2025.09.25 21:03:44.138138 [ 414 ] {} Context: ThreadFuzzer is enabled. Application will run slowly and unstable. 2025.09.25 21:03:44.139630 [ 414 ] {} Context: Server was built with address sanitizer. It will work slowly. 2025.09.25 21:03:44.664243 [ 414 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. 2025.09.25 21:03:44.672001 [ 414 ] {} Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. 2025.09.25 21:03:48.726875 [ 414 ] {} KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper. 2025.09.25 21:03:48.730037 [ 414 ] {} KeeperLogStore: Removing all changelogs 2025.09.25 21:03:48.809579 [ 414 ] {} KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647. 2025.09.25 21:03:48.819814 [ 414 ] {} RaftInstance: invalid election timeout lower bound detected, adjusted to 0 2025.09.25 21:03:48.820240 [ 414 ] {} RaftInstance: invalid election timeout upper bound detected, adjusted to 0 2025.09.25 21:03:48.876381 [ 1027 ] {} RaftInstance: Election timeout, initiate leader election 2025.09.25 21:03:49.451025 [ 414 ] {} Access(local_directory): File /var/lib/clickhouse/access/users.list doesn't exist 2025.09.25 21:03:49.451646 [ 414 ] {} Access(local_directory): Recovering lists in directory /var/lib/clickhouse/access/ 2025.09.25 21:03:50.045288 [ 414 ] {} Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring. 2025.09.25 21:03:54.223536 [ 417 ] {8ef43aed-173e-4842-a0ab-852ea7600abf} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE hits_v1; to attach existing table or CREATE TABLE hits_v1 ; to create new table or ATTACH TABLE hits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2025.09.25 21:04:07.760402 [ 417 ] {06f5ebcb-c628-4072-b560-2c30f04bd488} InterpreterCreateQuery: ATTACH TABLE query with full table definition is not recommended: use either ATTACH TABLE visits_v1; to attach existing table or CREATE TABLE visits_v1
; to create new table or ATTACH TABLE visits_v1 FROM '/path/to/data/'
; to create new table and attach data. 2025.09.25 21:04:20.427759 [ 1029 ] {} RaftInstance: failed to accept a rpc connection due to error 125, Operation canceled 2025.09.25 21:04:25.458671 [ 1275 ] {} Context: ThreadFuzzer is enabled. Application will run slowly and unstable. 2025.09.25 21:04:25.460032 [ 1275 ] {} Context: Server was built with address sanitizer. It will work slowly. 2025.09.25 21:04:25.952173 [ 1275 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. 2025.09.25 21:04:25.960509 [ 1275 ] {} Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. 2025.09.25 21:04:32.931489 [ 1275 ] {} KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647. 2025.09.25 21:04:32.948864 [ 1275 ] {} RaftInstance: invalid election timeout lower bound detected, adjusted to 0 2025.09.25 21:04:32.949647 [ 1275 ] {} RaftInstance: invalid election timeout upper bound detected, adjusted to 0 2025.09.25 21:04:33.010229 [ 1906 ] {} RaftInstance: Election timeout, initiate leader election 2025.09.25 21:04:40.948672 [ 1275 ] {} Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring. 2025.09.25 21:05:36.393247 [ 1278 ] {7a640ae9-5088-4032-897b-16b55efc0ad5} executeQuery: Code: 241. DB::Exception: Memory limit (for user) exceeded: would use 9.18 GiB (attempt to allocate chunk of 3635641 bytes), maximum: 9.18 GiB. OvercommitTracker decision: Query was selected to stop by OvercommitTracker. (MEMORY_LIMIT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:56552) (in query: INSERT INTO test.hits_s3 SELECT * FROM datasets.hits_v1 SETTINGS enable_filesystem_cache_on_write_operations=0, max_insert_threads=16), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>>(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity>>::type>, char const*&&, char const*&&, String&&, long&, String&&, char const*&&, std::basic_string_view>&&) @ 0x000000001adfc65d 4. ./build_docker/./src/Common/MemoryTracker.cpp:343: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfaddf 5. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfa777 6. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfa777 7. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:59: CurrentMemoryTracker::alloc(long) @ 0x000000001ad3289d 8. ./build_docker/./src/Common/Allocator.cpp:0: Allocator::realloc(void*, unsigned long, unsigned long, unsigned long) @ 0x000000001ad2f3ff 9. ./src/Common/PODArray.h:154: DB::ColumnFixedString::reserve(unsigned long) @ 0x000000002adfee0e 10. ./contrib/llvm-project/libcxx/include/vector:1491: DB::Squashing::squash(std::vector>&&, DB::CollectionOfDerivedItems&&) @ 0x00000000296dc8c3 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Squashing::squash(DB::Chunk&&) @ 0x00000000296db59b 12. ./src/Processors/Transforms/ApplySquashingTransform.h:35: DB::ApplySquashingTransform::onConsume(DB::Chunk) @ 0x00000000293628c9 13. ./contrib/llvm-project/libcxx/include/vector:432: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d9c1a3a 14. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function, DB::ThreadStatus*, std::atomic*) @ 0x000000002d9c1551 15. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d9c06a6 16. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 17. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 18. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 19. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 20. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007fb23ef79ac3 25. ? @ 0x00007fb23f00b850 2025.09.25 21:05:46.054552 [ 1278 ] {7a640ae9-5088-4032-897b-16b55efc0ad5} TCPHandler: Code: 241. DB::Exception: Memory limit (for user) exceeded: would use 9.18 GiB (attempt to allocate chunk of 3635641 bytes), maximum: 9.18 GiB. OvercommitTracker decision: Query was selected to stop by OvercommitTracker. (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>>(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity>>::type>, char const*&&, char const*&&, String&&, long&, String&&, char const*&&, std::basic_string_view>&&) @ 0x000000001adfc65d 4. ./build_docker/./src/Common/MemoryTracker.cpp:343: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfaddf 5. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfa777 6. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfa777 7. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:59: CurrentMemoryTracker::alloc(long) @ 0x000000001ad3289d 8. ./build_docker/./src/Common/Allocator.cpp:0: Allocator::realloc(void*, unsigned long, unsigned long, unsigned long) @ 0x000000001ad2f3ff 9. ./src/Common/PODArray.h:154: DB::ColumnFixedString::reserve(unsigned long) @ 0x000000002adfee0e 10. ./contrib/llvm-project/libcxx/include/vector:1491: DB::Squashing::squash(std::vector>&&, DB::CollectionOfDerivedItems&&) @ 0x00000000296dc8c3 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Squashing::squash(DB::Chunk&&) @ 0x00000000296db59b 12. ./src/Processors/Transforms/ApplySquashingTransform.h:35: DB::ApplySquashingTransform::onConsume(DB::Chunk) @ 0x00000000293628c9 13. ./contrib/llvm-project/libcxx/include/vector:432: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d9c1a3a 14. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function, DB::ThreadStatus*, std::atomic*) @ 0x000000002d9c1551 15. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d9c06a6 16. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 17. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 18. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 19. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 20. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007fb23ef79ac3 25. ? @ 0x00007fb23f00b850 2025.09.25 21:06:07.973366 [ 1278 ] {7b8b6d35-43cc-44b9-a268-ff8dd1030e0c} executeQuery: Code: 241. DB::Exception: Memory limit (for user) exceeded: would use 9.18 GiB (attempt to allocate chunk of 1066385 bytes), maximum: 9.18 GiB. OvercommitTracker decision: Query was selected to stop by OvercommitTracker.: (while reading column GoalsReached): (while reading from part store/78e/78ebf6a1-d987-4579-b3ec-00c1a087b1f3/201403_1_1_2/ in table datasets.hits_v1 (78ebf6a1-d987-4579-b3ec-00c1a087b1f3) located on disk __tmp_internal_262634248876485800814430204900764312493 of type web, from mark 20 with max_rows_to_read = 8192): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (MEMORY_LIMIT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50450) (in query: INSERT INTO test.hits SELECT * FROM datasets.hits_v1 SETTINGS enable_filesystem_cache_on_write_operations=0, max_insert_threads=16), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>>(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity>>::type>, char const*&&, char const*&&, String&&, long&, String&&, char const*&&, std::basic_string_view>&&) @ 0x000000001adfc65d 4. ./build_docker/./src/Common/MemoryTracker.cpp:343: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfaddf 5. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfa777 6. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfa777 7. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:59: CurrentMemoryTracker::alloc(long) @ 0x000000001ad3289d 8. ./build_docker/./src/Common/Allocator.cpp:151: Allocator::alloc(unsigned long, unsigned long) @ 0x000000001ad2eb85 9. void DB::PODArrayBase<8ul, 4096ul, Allocator, 63ul, 64ul>::resize<>(unsigned long) @ 0x000000000aa93cd0 10. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:0: DB::SerializationArray::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027ab99f0 11. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:196: DB::MergeTreeReaderWide::readData(DB::NameAndTypePair const&, std::shared_ptr const&, COW::immutable_ptr&, unsigned long, bool, unsigned long, unsigned long, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>&, bool) @ 0x000000002c80211e 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:158: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c800e8d 13. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 14. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000002c7e0473 15. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbec3 16. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 17. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:38: DB::MergeTreeThreadSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002de66094 18. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 19. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 20. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 21. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 25. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007fb23ef79ac3 30. ? @ 0x00007fb23f00b850 2025.09.25 21:06:17.496552 [ 1278 ] {7b8b6d35-43cc-44b9-a268-ff8dd1030e0c} TCPHandler: Code: 241. DB::Exception: Memory limit (for user) exceeded: would use 9.18 GiB (attempt to allocate chunk of 1066385 bytes), maximum: 9.18 GiB. OvercommitTracker decision: Query was selected to stop by OvercommitTracker.: (while reading column GoalsReached): (while reading from part store/78e/78ebf6a1-d987-4579-b3ec-00c1a087b1f3/201403_1_1_2/ in table datasets.hits_v1 (78ebf6a1-d987-4579-b3ec-00c1a087b1f3) located on disk __tmp_internal_262634248876485800814430204900764312493 of type web, from mark 20 with max_rows_to_read = 8192): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>>(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity>>::type>, char const*&&, char const*&&, String&&, long&, String&&, char const*&&, std::basic_string_view>&&) @ 0x000000001adfc65d 4. ./build_docker/./src/Common/MemoryTracker.cpp:343: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfaddf 5. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfa777 6. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfa777 7. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:59: CurrentMemoryTracker::alloc(long) @ 0x000000001ad3289d 8. ./build_docker/./src/Common/Allocator.cpp:151: Allocator::alloc(unsigned long, unsigned long) @ 0x000000001ad2eb85 9. void DB::PODArrayBase<8ul, 4096ul, Allocator, 63ul, 64ul>::resize<>(unsigned long) @ 0x000000000aa93cd0 10. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:0: DB::SerializationArray::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027ab99f0 11. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:196: DB::MergeTreeReaderWide::readData(DB::NameAndTypePair const&, std::shared_ptr const&, COW::immutable_ptr&, unsigned long, bool, unsigned long, unsigned long, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>&, bool) @ 0x000000002c80211e 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:158: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c800e8d 13. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 14. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000002c7e0473 15. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbec3 16. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 17. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:38: DB::MergeTreeThreadSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002de66094 18. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 19. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 20. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 21. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 25. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007fb23ef79ac3 30. ? @ 0x00007fb23f00b850 2025.09.25 21:07:04.938447 [ 1942 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {02260661-95d0-4eca-a73d-2543d99ce5b9::202509_1_41_8}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001adfc17a 4. ./build_docker/./src/Common/MemoryTracker.cpp:187: MemoryTracker::injectFault() const @ 0x000000001adf8d1a 5. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x000000002cc30a65 6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f47b 7. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007fb23ef79ac3 15. ? @ 0x00007fb23f00b850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:07:08.832166 [ 1936 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {95e6584c-1281-4071-8ae7-ba1b29088860::202509_1_21_4}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001adfc17a 4. ./build_docker/./src/Common/MemoryTracker.cpp:187: MemoryTracker::injectFault() const @ 0x000000001adf8d1a 5. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x000000002cc30a65 6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f47b 7. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007fb23ef79ac3 15. ? @ 0x00007fb23f00b850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:07:09.807680 [ 1909 ] {} RaftInstance: failed to accept a rpc connection due to error 125, Operation canceled 2025.09.25 21:07:12.874885 [ 2702 ] {} Context: ThreadFuzzer is enabled. Application will run slowly and unstable. 2025.09.25 21:07:12.876200 [ 2702 ] {} Context: Server was built with address sanitizer. It will work slowly. 2025.09.25 21:07:13.215372 [ 2702 ] {} Context: Server logging level is set to 'test' and performance is degraded. This cannot be used in production. 2025.09.25 21:07:13.217384 [ 2702 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. 2025.09.25 21:07:13.223053 [ 2702 ] {} Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. 2025.09.25 21:07:18.482646 [ 2702 ] {} KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647. 2025.09.25 21:07:18.530943 [ 2702 ] {} RaftInstance: invalid election timeout lower bound detected, adjusted to 0 2025.09.25 21:07:18.531915 [ 2702 ] {} RaftInstance: invalid election timeout upper bound detected, adjusted to 0 2025.09.25 21:07:18.595770 [ 3328 ] {} RaftInstance: Election timeout, initiate leader election 2025.09.25 21:07:27.104854 [ 2702 ] {} Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring. 2025.09.25 21:07:29.664656 [ 2705 ] {f5cf4fd2-2bf5-4c3f-aab6-5c74f77a0cac} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:49526) (in query: SELECT count() FROM system.build_options WHERE name = 'CXX_FLAGS' AND position('sanitize=address' IN value)), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:29.673335 [ 2705 ] {f5cf4fd2-2bf5-4c3f-aab6-5c74f77a0cac} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:35.993436 [ 2707 ] {78bc2fd1-e663-4943-aee9-afaf460d6608} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:49842) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:306: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349fbb 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:35.995982 [ 2707 ] {78bc2fd1-e663-4943-aee9-afaf460d6608} DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:306: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349fbb 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:07:36.905438 [ 3619 ] {8a395857-cfac-49b4-9ef8-16cb8d03d4fb} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42306) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:113: DB::CompletedPipelineExecutor::execute() @ 0x000000002d34579e 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1635: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b40110 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:36.908798 [ 3619 ] {8a395857-cfac-49b4-9ef8-16cb8d03d4fb} DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:113: DB::CompletedPipelineExecutor::execute() @ 0x000000002d34579e 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1635: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b40110 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:07:39.044797 [ 2707 ] {d94feb68-c694-4238-87ec-a4a051856532} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42484) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:113: DB::CompletedPipelineExecutor::execute() @ 0x000000002d34579e 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1635: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b40110 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:39.051814 [ 2707 ] {d94feb68-c694-4238-87ec-a4a051856532} DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=8, jobs=8). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:113: DB::CompletedPipelineExecutor::execute() @ 0x000000002d34579e 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1635: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b40110 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:07:39.710366 [ 3619 ] {fa03b7c4-6e8f-4671-a6c1-1ba6e4abbf0c} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42540) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:113: DB::CompletedPipelineExecutor::execute() @ 0x000000002d34579e 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1635: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b40110 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:39.724312 [ 3619 ] {fa03b7c4-6e8f-4671-a6c1-1ba6e4abbf0c} DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:113: DB::CompletedPipelineExecutor::execute() @ 0x000000002d34579e 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1635: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b40110 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:07:40.044971 [ 2707 ] {982d0ee3-2ec9-4594-839a-c45026aab5bd} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42580) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:113: DB::CompletedPipelineExecutor::execute() @ 0x000000002d34579e 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1635: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b40110 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:40.060866 [ 2707 ] {982d0ee3-2ec9-4594-839a-c45026aab5bd} DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:113: DB::CompletedPipelineExecutor::execute() @ 0x000000002d34579e 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1635: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b40110 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:07:41.112141 [ 3619 ] {48b65ad3-dc13-49d7-8859-65fdec2c47cd} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42640) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:306: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349fbb 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:41.123127 [ 3619 ] {48b65ad3-dc13-49d7-8859-65fdec2c47cd} DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:306: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349fbb 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:07:41.673163 [ 3656 ] {8c686d30-85d0-4b2f-836c-2fffe40caf82} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42680) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:306: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349fbb 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:41.690502 [ 3656 ] {8c686d30-85d0-4b2f-836c-2fffe40caf82} DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:306: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349fbb 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:07:43.228503 [ 3619 ] {46d53a63-9372-470c-a8af-04eec94ad325} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42764) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:306: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349fbb 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:43.230541 [ 3619 ] {46d53a63-9372-470c-a8af-04eec94ad325} DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:306: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349fbb 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:07:43.622203 [ 2705 ] {b443dc0d-5956-45bc-898c-747021222a4f} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42856) (in query: SELECT version(), arrayStringConcat(groupArray(value), ' ') FROM system.build_options WHERE name IN ('GIT_HASH', 'GIT_BRANCH') ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:306: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349fbb 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:43.656718 [ 2705 ] {b443dc0d-5956-45bc-898c-747021222a4f} DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:306: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349fbb 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:07:44.395469 [ 3648 ] {c6fce33d-7562-4fd8-bc5d-6d915ecc5459} executeQuery: Code: 258. DB::Exception: Different number of columns in UNION_ALL elements: c1, c2, c3 and c1, c2. (UNION_ALL_RESULT_STRUCTURES_MISMATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43648) (comment: 01036_union_different_columns.sql) (in query: select 1 as c1, 2 as c2, 3 as c3 union all (select 1 as c1, 2 as c2, 3 as c3 union all select 1 as c1, 2 as c2) -- { serverError UNION_ALL_RESULT_STRUCTURES_MISMATCH } ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&&, String&&, String&&) @ 0x00000000282f38a0 4. ./build_docker/./src/Planner/Utils.cpp:95: DB::buildCommonHeaderForUnion(std::vector> const&, DB::SelectUnionMode) @ 0x00000000282ec243 5. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForUnionNode() @ 0x0000000029427615 6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261f4 7. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:44.398591 [ 3648 ] {c6fce33d-7562-4fd8-bc5d-6d915ecc5459} TCPHandler: Code: 258. DB::Exception: Different number of columns in UNION_ALL elements: c1, c2, c3 and c1, c2. (UNION_ALL_RESULT_STRUCTURES_MISMATCH), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, char const*&&, String&&, String&&) @ 0x00000000282f38a0 4. ./build_docker/./src/Planner/Utils.cpp:95: DB::buildCommonHeaderForUnion(std::vector> const&, DB::SelectUnionMode) @ 0x00000000282ec243 5. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForUnionNode() @ 0x0000000029427615 6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261f4 7. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:47.442957 [ 3619 ] {dd8ef571-dbb3-4e45-952d-e7a56b86045f} executeQuery: Code: 28. DB::Exception: Too high precision requested for Float, must not be more than 60, got 61: In scope SELECT toDecimalString(CAST('32.32', 'Float32'), 61). (CANNOT_PRINT_FLOAT_OR_DOUBLE_NUMBER) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43632) (comment: 02676_to_decimal_string.sql) (in query: -- Max number of decimal fractional digits is defined as 77 for Int/UInt/Decimal and 60 for Float. -- These values shall work OK. SELECT toDecimalString('32.32'::Float32, 61);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<_BitInt(8)>(int, FormatStringHelperImpl::type>, std::type_identity<_BitInt(8)>&&) @ 0x0000000018621df4 4. _ZN2DB12_GLOBAL__N_123FunctionToDecimalString6formatITk17is_floating_pointfEEvT_RNS_11WriteBufferEDu @ 0x00000000186210e5 5. DB::(anonymous namespace)::FunctionToDecimalString::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000018612559 6. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 7. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 9. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 10. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 11. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 12. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 23. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 28. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 2025.09.25 21:07:47.443147 [ 2707 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-1} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55142) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:47.444971 [ 3619 ] {dd8ef571-dbb3-4e45-952d-e7a56b86045f} TCPHandler: Code: 28. DB::Exception: Too high precision requested for Float, must not be more than 60, got 61: In scope SELECT toDecimalString(CAST('32.32', 'Float32'), 61). (CANNOT_PRINT_FLOAT_OR_DOUBLE_NUMBER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<_BitInt(8)>(int, FormatStringHelperImpl::type>, std::type_identity<_BitInt(8)>&&) @ 0x0000000018621df4 4. _ZN2DB12_GLOBAL__N_123FunctionToDecimalString6formatITk17is_floating_pointfEEvT_RNS_11WriteBufferEDu @ 0x00000000186210e5 5. DB::(anonymous namespace)::FunctionToDecimalString::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000018612559 6. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 7. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 9. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 10. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 11. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 12. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 23. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 28. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 2025.09.25 21:07:47.456936 [ 2707 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-1} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:47.588486 [ 3619 ] {30e04a6c-30bf-4bb1-aa76-927b790a0d60} executeQuery: Code: 28. DB::Exception: Too high precision requested for Float, must not be more than 60, got 61: In scope SELECT toDecimalString(CAST('64.64', 'Float64'), 61). (CANNOT_PRINT_FLOAT_OR_DOUBLE_NUMBER) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43632) (comment: 02676_to_decimal_string.sql) (in query: SELECT toDecimalString('64.64'::Float64, 61);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<_BitInt(8)>(int, FormatStringHelperImpl::type>, std::type_identity<_BitInt(8)>&&) @ 0x0000000018621df4 4. _ZN2DB12_GLOBAL__N_123FunctionToDecimalString6formatITk17is_floating_pointdEEvT_RNS_11WriteBufferEDu @ 0x0000000018621585 5. DB::(anonymous namespace)::FunctionToDecimalString::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x00000000186130a0 6. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 7. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 9. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 10. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 11. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 12. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 23. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 28. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 2025.09.25 21:07:47.598705 [ 3619 ] {30e04a6c-30bf-4bb1-aa76-927b790a0d60} TCPHandler: Code: 28. DB::Exception: Too high precision requested for Float, must not be more than 60, got 61: In scope SELECT toDecimalString(CAST('64.64', 'Float64'), 61). (CANNOT_PRINT_FLOAT_OR_DOUBLE_NUMBER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<_BitInt(8)>(int, FormatStringHelperImpl::type>, std::type_identity<_BitInt(8)>&&) @ 0x0000000018621df4 4. _ZN2DB12_GLOBAL__N_123FunctionToDecimalString6formatITk17is_floating_pointdEEvT_RNS_11WriteBufferEDu @ 0x0000000018621585 5. DB::(anonymous namespace)::FunctionToDecimalString::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x00000000186130a0 6. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 7. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 9. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 10. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 11. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 12. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 23. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 28. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 2025.09.25 21:07:47.862778 [ 3619 ] {01caa8ea-bf96-4ba5-8e69-5b09b704c72a} executeQuery: Code: 28. DB::Exception: Too many fractional digits requested, shall not be more than 77: In scope SELECT toDecimalString(CAST('88', 'UInt8'), 78). (CANNOT_PRINT_FLOAT_OR_DOUBLE_NUMBER) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43632) (comment: 02676_to_decimal_string.sql) (in query: SELECT toDecimalString('88'::UInt8, 78);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, unsigned long const&) @ 0x000000000cbb00f4 4. DB::(anonymous namespace)::FunctionToDecimalString::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000001861736d 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:07:47.867181 [ 3619 ] {01caa8ea-bf96-4ba5-8e69-5b09b704c72a} TCPHandler: Code: 28. DB::Exception: Too many fractional digits requested, shall not be more than 77: In scope SELECT toDecimalString(CAST('88', 'UInt8'), 78). (CANNOT_PRINT_FLOAT_OR_DOUBLE_NUMBER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, unsigned long const&) @ 0x000000000cbb00f4 4. DB::(anonymous namespace)::FunctionToDecimalString::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000001861736d 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:07:47.972098 [ 3627 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-2} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-2' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55182) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:47.992770 [ 3627 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-2} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-2' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:48.093844 [ 3619 ] {8503e728-703b-42cc-b194-797904250487} executeQuery: Code: 28. DB::Exception: Too many fractional digits requested, shall not be more than 77: In scope SELECT toDecimalString(CAST('646464', 'Int256'), 78). (CANNOT_PRINT_FLOAT_OR_DOUBLE_NUMBER) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43632) (comment: 02676_to_decimal_string.sql) (in query: SELECT toDecimalString('646464'::Int256, 78);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, unsigned long const&) @ 0x000000000cbb00f4 4. DB::(anonymous namespace)::FunctionToDecimalString::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000018616f7c 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:07:48.149215 [ 3619 ] {8503e728-703b-42cc-b194-797904250487} TCPHandler: Code: 28. DB::Exception: Too many fractional digits requested, shall not be more than 77: In scope SELECT toDecimalString(CAST('646464', 'Int256'), 78). (CANNOT_PRINT_FLOAT_OR_DOUBLE_NUMBER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, unsigned long const&) @ 0x000000000cbb00f4 4. DB::(anonymous namespace)::FunctionToDecimalString::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000018616f7c 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:07:48.235863 [ 3656 ] {8d8b3767-5993-4637-896e-8a94f7476014} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=2, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43686) (comment: 01213_alter_rename_primary_key_zookeeper_long.sql) (in query: SELECT key1, value1 FROM table_for_rename_pk WHERE key1 = 1 AND key2 = 1 AND key3 = 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:48.251200 [ 3656 ] {8d8b3767-5993-4637-896e-8a94f7476014} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=2, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:48.360329 [ 3619 ] {94c63d1d-e91d-42bf-9ea6-f26f4b1e5019} executeQuery: Code: 28. DB::Exception: Too many fractional digits requested for Decimal, must not be more than 77: In scope SELECT toDecimalString(CAST('-128.789323123321329854641231237893231233213298546', 'Decimal256(45)'), 78). (CANNOT_PRINT_FLOAT_OR_DOUBLE_NUMBER) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43632) (comment: 02676_to_decimal_string.sql) (in query: SELECT toDecimalString('-128.789323123321329854641231237893231233213298546'::Decimal256(45), 78);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, unsigned long const&) @ 0x000000000cbb00f4 4. DB::(anonymous namespace)::FunctionToDecimalString::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000001861775b 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:07:48.364247 [ 3619 ] {94c63d1d-e91d-42bf-9ea6-f26f4b1e5019} TCPHandler: Code: 28. DB::Exception: Too many fractional digits requested for Decimal, must not be more than 77: In scope SELECT toDecimalString(CAST('-128.789323123321329854641231237893231233213298546', 'Decimal256(45)'), 78). (CANNOT_PRINT_FLOAT_OR_DOUBLE_NUMBER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, unsigned long const&) @ 0x000000000cbb00f4 4. DB::(anonymous namespace)::FunctionToDecimalString::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000001861775b 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:07:48.414612 [ 2707 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-3} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-3' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55194) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:48.438972 [ 2707 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-3} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-3' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:48.554169 [ 3619 ] {7c3fd930-62a9-4f93-9ec6-09182457bf59} executeQuery: Code: 44. DB::Exception: A value of illegal type was provided as 2nd argument 'precision' to function 'toDecimalString'. Expected: const Integer, got: UInt8: In scope SELECT toDecimalString(CAST('256.256', 'Decimal256(45)'), *). (ILLEGAL_COLUMN) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43632) (comment: 02676_to_decimal_string.sql) (in query: -- wrong types: #52407 and similar SELECT toDecimalString('256.256'::Decimal256(45), *);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception> const&, String, std::basic_string_view> const&, String>(int, FormatStringHelperImpl::type, std::type_identity> const&>::type, std::type_identity::type, std::type_identity> const&>::type, std::type_identity::type>, String&&, std::basic_string_view> const&, String&&, std::basic_string_view> const&, String&&) @ 0x000000000aaec6f5 4. DB::(anonymous namespace)::validateArgumentsImpl(DB::IFunction const&, std::vector> const&, unsigned long, std::vector> const&) @ 0x000000000aae6b1a 5. DB::validateFunctionArguments(DB::IFunction const&, std::vector> const&, std::vector> const&, std::vector> const&) @ 0x000000000aae5e2a 6. DB::(anonymous namespace)::FunctionToDecimalString::getReturnTypeImpl(std::vector> const&) const @ 0x000000001861efc2 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:07:48.571590 [ 3619 ] {7c3fd930-62a9-4f93-9ec6-09182457bf59} TCPHandler: Code: 44. DB::Exception: A value of illegal type was provided as 2nd argument 'precision' to function 'toDecimalString'. Expected: const Integer, got: UInt8: In scope SELECT toDecimalString(CAST('256.256', 'Decimal256(45)'), *). (ILLEGAL_COLUMN), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception> const&, String, std::basic_string_view> const&, String>(int, FormatStringHelperImpl::type, std::type_identity> const&>::type, std::type_identity::type, std::type_identity> const&>::type, std::type_identity::type>, String&&, std::basic_string_view> const&, String&&, std::basic_string_view> const&, String&&) @ 0x000000000aaec6f5 4. DB::(anonymous namespace)::validateArgumentsImpl(DB::IFunction const&, std::vector> const&, unsigned long, std::vector> const&) @ 0x000000000aae6b1a 5. DB::validateFunctionArguments(DB::IFunction const&, std::vector> const&, std::vector> const&, std::vector> const&) @ 0x000000000aae5e2a 6. DB::(anonymous namespace)::FunctionToDecimalString::getReturnTypeImpl(std::vector> const&) const @ 0x000000001861efc2 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:07:48.727477 [ 3619 ] {b03f19a4-06d7-433a-a7f4-959315b177b2} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'precision' to function 'toDecimalString'. Expected: const Integer, got: String: In scope SELECT toDecimalString(CAST('128.128', 'Decimal128(30)'), 'str'). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43632) (comment: 02676_to_decimal_string.sql) (in query: SELECT toDecimalString('128.128'::Decimal128(30), 'str');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception> const&, String, std::basic_string_view> const&, String>(int, FormatStringHelperImpl::type, std::type_identity> const&>::type, std::type_identity::type, std::type_identity> const&>::type, std::type_identity::type>, String&&, std::basic_string_view> const&, String&&, std::basic_string_view> const&, String&&) @ 0x000000000aaec6f5 4. DB::(anonymous namespace)::validateArgumentsImpl(DB::IFunction const&, std::vector> const&, unsigned long, std::vector> const&) @ 0x000000000aae6b1a 5. DB::validateFunctionArguments(DB::IFunction const&, std::vector> const&, std::vector> const&, std::vector> const&) @ 0x000000000aae5e2a 6. DB::(anonymous namespace)::FunctionToDecimalString::getReturnTypeImpl(std::vector> const&) const @ 0x000000001861efc2 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:07:48.735634 [ 3619 ] {b03f19a4-06d7-433a-a7f4-959315b177b2} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'precision' to function 'toDecimalString'. Expected: const Integer, got: String: In scope SELECT toDecimalString(CAST('128.128', 'Decimal128(30)'), 'str'). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception> const&, String, std::basic_string_view> const&, String>(int, FormatStringHelperImpl::type, std::type_identity> const&>::type, std::type_identity::type, std::type_identity> const&>::type, std::type_identity::type>, String&&, std::basic_string_view> const&, String&&, std::basic_string_view> const&, String&&) @ 0x000000000aaec6f5 4. DB::(anonymous namespace)::validateArgumentsImpl(DB::IFunction const&, std::vector> const&, unsigned long, std::vector> const&) @ 0x000000000aae6b1a 5. DB::validateFunctionArguments(DB::IFunction const&, std::vector> const&, std::vector> const&, std::vector> const&) @ 0x000000000aae5e2a 6. DB::(anonymous namespace)::FunctionToDecimalString::getReturnTypeImpl(std::vector> const&) const @ 0x000000001861efc2 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:07:48.917059 [ 2707 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-4} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-4' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55210) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:48.926195 [ 2707 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-4} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-4' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:48.970151 [ 3619 ] {9f12a7f4-c4fe-4bf8-968c-decb8225f8c2} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDecimalString doesn't match: passed 1, should be 2: In scope SELECT toDecimalString(CAST('64.64', 'Decimal64(10)')). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43632) (comment: 02676_to_decimal_string.sql) (in query: SELECT toDecimalString('64.64'::Decimal64(10));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, unsigned long&, unsigned long&) @ 0x000000000ccd84e0 4. DB::IFunctionOverloadResolver::checkNumberOfArguments(unsigned long) const @ 0x000000000ccd2a84 5. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd35cf 6. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 7. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 8. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 18. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 19. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 21. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 23. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 27. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007f12cd75eac3 30. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:48.978997 [ 3619 ] {9f12a7f4-c4fe-4bf8-968c-decb8225f8c2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDecimalString doesn't match: passed 1, should be 2: In scope SELECT toDecimalString(CAST('64.64', 'Decimal64(10)')). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, unsigned long&, unsigned long&) @ 0x000000000ccd84e0 4. DB::IFunctionOverloadResolver::checkNumberOfArguments(unsigned long) const @ 0x000000000ccd2a84 5. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd35cf 6. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 7. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 8. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 18. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 19. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 21. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 23. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 27. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007f12cd75eac3 30. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:49.078460 [ 3619 ] {0ff43f97-895d-4c95-b1bb-b43e2baae8b9} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDecimalString doesn't match: passed 3, should be 2: In scope SELECT toDecimalString(CAST('64.64', 'Decimal64(10)'), 3, 3). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43632) (comment: 02676_to_decimal_string.sql) (in query: SELECT toDecimalString('64.64'::Decimal64(10), 3, 3);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, unsigned long&, unsigned long&) @ 0x000000000ccd84e0 4. DB::IFunctionOverloadResolver::checkNumberOfArguments(unsigned long) const @ 0x000000000ccd2a84 5. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd35cf 6. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 7. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 8. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 18. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 19. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 21. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 23. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 27. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007f12cd75eac3 30. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:49.089324 [ 3619 ] {0ff43f97-895d-4c95-b1bb-b43e2baae8b9} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDecimalString doesn't match: passed 3, should be 2: In scope SELECT toDecimalString(CAST('64.64', 'Decimal64(10)'), 3, 3). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, unsigned long&, unsigned long&) @ 0x000000000ccd84e0 4. DB::IFunctionOverloadResolver::checkNumberOfArguments(unsigned long) const @ 0x000000000ccd2a84 5. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd35cf 6. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 7. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 8. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 18. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 19. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 21. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 23. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 27. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007f12cd75eac3 30. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:49.191510 [ 3851 ] {8c169c09-0655-48fa-b2ec-84c601145f9e} executeQuery: Code: 503. DB::Exception: Aggregate function aggThrow has thrown exception successfully: while executing 'FUNCTION arrayReduce('aggThrow(0.0001)'_String :: 1, range(modulo(__table1.number, 10_UInt8)) :: 2) -> arrayReduce('aggThrow(0.0001)'_String, range(modulo(__table1.number, 10_UInt8))) UInt8 : 0'. (AGGREGATE_FUNCTION_THROW) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55222) (comment: 01052_array_reduce_exception.sql) (in query: SELECT arrayReduce('aggThrow(0.0001)', range(number % 10)) FROM system.numbers FORMAT Null;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/AggregateFunctions/AggregateFunctionAggThrow.cpp:71: DB::(anonymous namespace)::AggregateFunctionThrow::create(char*) const @ 0x0000000025a1b443 5. DB::FunctionArrayReduce::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000001a322481 6. DB::FunctionToExecutableFunctionAdaptor::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac9135 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccce5b 8. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce7e4 9. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 10. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::ExpressionActions::execute(DB::Block&, unsigned long&, bool, bool) const @ 0x00000000286fc2bd 11. ./build_docker/./src/Processors/Transforms/ExpressionTransform.cpp:0: DB::ExpressionTransform::transform(DB::Chunk&) @ 0x000000002d9c29ae 12. ./contrib/llvm-project/libcxx/include/__utility/swap.h:35: DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x000000001b49a31f 13. ./build_docker/./src/Processors/ISimpleTransform.cpp:99: DB::ISimpleTransform::work() @ 0x000000002d323859 14. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 15. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 18. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 19. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:49.193373 [ 3851 ] {8c169c09-0655-48fa-b2ec-84c601145f9e} TCPHandler: Code: 503. DB::Exception: Aggregate function aggThrow has thrown exception successfully: while executing 'FUNCTION arrayReduce('aggThrow(0.0001)'_String :: 1, range(modulo(__table1.number, 10_UInt8)) :: 2) -> arrayReduce('aggThrow(0.0001)'_String, range(modulo(__table1.number, 10_UInt8))) UInt8 : 0'. (AGGREGATE_FUNCTION_THROW), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/AggregateFunctions/AggregateFunctionAggThrow.cpp:71: DB::(anonymous namespace)::AggregateFunctionThrow::create(char*) const @ 0x0000000025a1b443 5. DB::FunctionArrayReduce::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000001a322481 6. DB::FunctionToExecutableFunctionAdaptor::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac9135 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccce5b 8. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce7e4 9. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 10. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::ExpressionActions::execute(DB::Block&, unsigned long&, bool, bool) const @ 0x00000000286fc2bd 11. ./build_docker/./src/Processors/Transforms/ExpressionTransform.cpp:0: DB::ExpressionTransform::transform(DB::Chunk&) @ 0x000000002d9c29ae 12. ./contrib/llvm-project/libcxx/include/__utility/swap.h:35: DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x000000001b49a31f 13. ./build_docker/./src/Processors/ISimpleTransform.cpp:99: DB::ISimpleTransform::work() @ 0x000000002d323859 14. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 15. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 18. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 19. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:49.550048 [ 3656 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-5} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55240) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:49.554839 [ 3656 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-5} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:50.010549 [ 3656 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-6} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-6' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55274) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:50.014147 [ 3656 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-6} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-6' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:50.603614 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-7} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-7' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55300) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:50.627834 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-7} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-7' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:51.037943 [ 3851 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-8} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55334) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:51.060150 [ 3851 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-8} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:51.562518 [ 3656 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-9} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55358) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:51.570595 [ 3656 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-9} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:52.459731 [ 2705 ] {023c14a0-db76-40ad-8967-c09d0e4215e1} executeQuery: Code: 62. DB::Exception: Cannot TRUNCATE dictionary. (SYNTAX_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55284) (comment: 00157_cache_dictionary.sql) (in query: DROP DICTIONARY IF EXISTS db_dict.cache_hits;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:262: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f1adc 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:52.492037 [ 2705 ] {023c14a0-db76-40ad-8967-c09d0e4215e1} TCPHandler: Code: 62. DB::Exception: Cannot TRUNCATE dictionary. (SYNTAX_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:262: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f1adc 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:52.675007 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-10} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55376) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:52.714696 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-10} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:53.434205 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-11} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55400) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:53.450278 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-11} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:54.065847 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-12} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-12' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55414) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:54.081403 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-12} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-12' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:54.113173 [ 3627 ] {0c11e846-74b3-4e06-94a0-6a93beba26b0} executeQuery: Code: 471. DB::Exception: Could not specify `disk` and `storage_policy` at the same time for storage Log Family. (INVALID_SETTING_VALUE) (version 24.8.14.10508.altinitytest (altinity build)) (from 0.0.0.0:0) (comment: 02554_log_faminy_support_storage_policy.sql) (in query: /* ddl_entry=query-0000000006 */ CREATE TABLE test_5.test_2554_error UUID '2b61d17b-1ac1-4bbf-89da-b2a999113120' (`n` UInt32) ENGINE = Log SETTINGS disk = 'default', storage_policy = 'default'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Storages/StorageLogSettings.cpp:27: DB::getDiskName(DB::ASTStorage&, std::shared_ptr) @ 0x000000002be2df53 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: std::shared_ptr std::__function::__policy_invoker (DB::StorageFactory::Arguments const&)>::__call_impl (DB::StorageFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::StorageFactory::Arguments const&) @ 0x000000002be1c547 6. ./build_docker/./src/Storages/StorageFactory.cpp:225: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x000000002b77bf53 7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1659: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291b9254 8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 9. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 12. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 14. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 15. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 17. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 18. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 19. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 21. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 22. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 24. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 25. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:54.168089 [ 3627 ] {0c11e846-74b3-4e06-94a0-6a93beba26b0} executeQuery: Code: 471. DB::Exception: Could not specify `disk` and `storage_policy` at the same time for storage Log Family. (INVALID_SETTING_VALUE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55250) (comment: 02554_log_faminy_support_storage_policy.sql) (in query: CREATE TABLE test_2554_error (n UInt32) ENGINE = Log SETTINGS disk = 'default', storage_policy = 'default';), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Storages/StorageLogSettings.cpp:27: DB::getDiskName(DB::ASTStorage&, std::shared_ptr) @ 0x000000002be2df53 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: std::shared_ptr std::__function::__policy_invoker (DB::StorageFactory::Arguments const&)>::__call_impl (DB::StorageFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::StorageFactory::Arguments const&) @ 0x000000002be1c547 6. ./build_docker/./src/Storages/StorageFactory.cpp:225: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x000000002b77bf53 7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1659: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291b9254 8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 9. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 12. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 14. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 15. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 17. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 18. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 19. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 21. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 22. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 24. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 25. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:54.173536 [ 3627 ] {0c11e846-74b3-4e06-94a0-6a93beba26b0} TCPHandler: Code: 471. DB::Exception: Could not specify `disk` and `storage_policy` at the same time for storage Log Family. (INVALID_SETTING_VALUE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Storages/StorageLogSettings.cpp:27: DB::getDiskName(DB::ASTStorage&, std::shared_ptr) @ 0x000000002be2df53 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: std::shared_ptr std::__function::__policy_invoker (DB::StorageFactory::Arguments const&)>::__call_impl (DB::StorageFactory::Arguments const&)>>(std::__function::__policy_storage const*, DB::StorageFactory::Arguments const&) @ 0x000000002be1c547 6. ./build_docker/./src/Storages/StorageFactory.cpp:225: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x000000002b77bf53 7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1659: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291b9254 8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 9. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 12. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 14. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 15. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 17. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 18. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 19. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 21. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 22. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 24. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 25. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:54.702219 [ 3656 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-13} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55448) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34820f 6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 9. asan_thread_start(void*) @ 0x000000000aa43059 10. ? @ 0x00007f12cd75eac3 11. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:54.723490 [ 3656 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-13} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34820f 6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 9. asan_thread_start(void*) @ 0x000000000aa43059 10. ? @ 0x00007f12cd75eac3 11. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:55.597964 [ 3619 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-14} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-14' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55478) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:55.614964 [ 3619 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-14} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-14' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:56.219599 [ 2707 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-15} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55532) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:56.275196 [ 2707 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-15} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:57.036567 [ 3619 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-16} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-16' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51596) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:57.088025 [ 3619 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-16} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-16' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:57.685335 [ 3619 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-17} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-17' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51606) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:57.702690 [ 3619 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-17} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-17' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:58.231067 [ 3619 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-18} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-18' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51640) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:58.258844 [ 3619 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-18} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-18' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:59.509437 [ 2707 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-19} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51650) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:07:59.513879 [ 2707 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-19} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:00.083472 [ 3656 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-20} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51692) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:00.092244 [ 3656 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-20} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:00.620866 [ 3851 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-21} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51722) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34820f 6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 9. asan_thread_start(void*) @ 0x000000000aa43059 10. ? @ 0x00007f12cd75eac3 11. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:00.653174 [ 3851 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-21} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34820f 6. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 9. asan_thread_start(void*) @ 0x000000000aa43059 10. ? @ 0x00007f12cd75eac3 11. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:00.944413 [ 3648 ] {a806222b-0181-4022-a581-a9b11ae464ad} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51702) (comment: 00156_max_execution_speed_sample_merge.sql) (in query: SELECT count() FROM test.hits SAMPLE 1 / 2;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:00.949144 [ 3648 ] {a806222b-0181-4022-a581-a9b11ae464ad} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:01.185164 [ 3851 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-22} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-22' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51748) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:01.222565 [ 3851 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-22} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-22' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:01.642814 [ 3656 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-23} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51794) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:01.648596 [ 3656 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-23} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:02.529633 [ 3851 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-24} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51816) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:02.547198 [ 3851 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-24} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:02.702832 [ 3619 ] {deedd3f1-788e-4cce-a658-979c61448abf} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51768) (comment: 00072_compare_date_and_string_index.sql) (in query: SELECT count() FROM test.hits WHERE EventDate = toDate('2014-03-18');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:02.709781 [ 3619 ] {deedd3f1-788e-4cce-a658-979c61448abf} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:03.234506 [ 3619 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-25} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51866) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:03.250514 [ 3619 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-25} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:03.830921 [ 3656 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-26} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-26' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51874) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:03.848600 [ 3656 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-26} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-26' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:04.659371 [ 2707 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-27} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-27' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51920) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:04.683624 [ 2707 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-27} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-27' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:05.509473 [ 3816 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-28} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-28' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51964) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:05.514483 [ 3816 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-28} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-28' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:06.360912 [ 3682 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-29} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51984) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:454: DB::QueryStatus::addPipelineExecutor(DB::PipelineExecutor*) @ 0x00000000295fb4fc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::PipelineExecutor(std::shared_ptr, std::allocator>>>&, std::shared_ptr) @ 0x000000002d34766a 6. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000002d368e67 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:06.387520 [ 3682 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-29} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:454: DB::QueryStatus::addPipelineExecutor(DB::PipelineExecutor*) @ 0x00000000295fb4fc 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::PipelineExecutor(std::shared_ptr, std::allocator>>>&, std::shared_ptr) @ 0x000000002d34766a 6. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:0: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x000000002d368e67 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:06.942496 [ 3816 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-30} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-30' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:45984) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:06.954410 [ 3816 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-30} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-30' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:07.539884 [ 2707 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-31} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-31' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46038) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:07.566448 [ 2707 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-31} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-31' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:08.122327 [ 3656 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-32} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-32' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46044) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:08.145573 [ 3656 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-32} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-32' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:08.619280 [ 3682 ] {7feff791-300a-422a-a535-ea5d2fa115c2} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46078) (comment: 02125_lz4_compression_bug_Native.sh) (in query: SELECT max(x) FROM file), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:08.629530 [ 3682 ] {7feff791-300a-422a-a535-ea5d2fa115c2} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:09.013802 [ 3627 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-33} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46090) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:09.023820 [ 3627 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-33} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:09.688393 [ 2705 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-34} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-34' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46152) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:09.694261 [ 2705 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-34} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-34' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:10.069415 [ 3619 ] {4b51a5bc-441b-455d-8547-14867865040c} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51872) (comment: 00072_compare_date_and_string_index.sql) (in query: SELECT count() FROM test.hits_indexed_by_time WHERE EventTime < toDateTime('2014-03-18 01:02:03', 'Asia/Dubai');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:10.080396 [ 3619 ] {4b51a5bc-441b-455d-8547-14867865040c} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:10.443886 [ 3682 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-35} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=730, jobs=713). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46200) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:10.450205 [ 3682 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-35} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=730, jobs=713). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:11.120843 [ 2705 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-36} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46242) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:11.127341 [ 2705 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-36} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:11.193684 [ 3682 ] {e6c18695-c46e-432a-b639-261fc8a304ec} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46226) (comment: 02125_lz4_compression_bug_Native.sh) (in query: SELECT max(x) FROM file), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:11.198586 [ 3682 ] {e6c18695-c46e-432a-b639-261fc8a304ec} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:11.799115 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-37} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-37' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46292) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:11.834208 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-37} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-37' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:12.421249 [ 2705 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-38} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46316) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:12.459233 [ 2705 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-38} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:12.996289 [ 3816 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-39} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46380) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:13.000155 [ 3816 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-39} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:13.460936 [ 2707 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-40} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-40' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46416) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:13.470310 [ 2707 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-40} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-40' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:14.002975 [ 3816 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-41} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-41' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46446) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:14.019185 [ 3816 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-41} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-41' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:14.503233 [ 3816 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-42} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46486) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:14.507021 [ 3816 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-42} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:14.667186 [ 3648 ] {75d9eb04-84df-454a-8fb0-6a22fb78b861} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46490) (comment: 02125_lz4_compression_bug_Native.sh) (in query: SELECT max(x) FROM file), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:14.679153 [ 3648 ] {75d9eb04-84df-454a-8fb0-6a22fb78b861} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=17, jobs=17). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:15.207045 [ 2705 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-43} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-43' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46502) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:15.210603 [ 2705 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-43} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-43' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:15.743960 [ 2705 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-44} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-44' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46524) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:15.758702 [ 2705 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-44} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-44' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:16.253282 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-45} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-45' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46568) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:16.259796 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-45} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-45' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:16.814243 [ 3816 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-46} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60250) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:16.820459 [ 3816 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-46} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:17.528945 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-47} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60284) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:17.547997 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-47} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:18.228461 [ 3816 ] {31d27b7a-9ca2-4926-99f2-bb4b46d7f7a2} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=746, jobs=716). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60296) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: kill query where query_id = '02450_kill_distributed_query_deadlock_test_j5v4sbbn-48' sync), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:18.250509 [ 3816 ] {31d27b7a-9ca2-4926-99f2-bb4b46d7f7a2} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=746, jobs=716). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:18.694863 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-48} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60310) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:18.805936 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-48} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:19.384657 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-49} executeQuery: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-49' is killed in pending state. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60338) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:19.394221 [ 3648 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-49} TCPHandler: Code: 394. DB::Exception: Query '02450_kill_distributed_query_deadlock_test_j5v4sbbn-49' is killed in pending state. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/executeQuery.cpp:1294: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b38b88 5. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 7. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 8. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 10. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 11. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:19.833621 [ 2705 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-50} executeQuery: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60340) (comment: 02450_kill_distributed_query_deadlock.sh) (in query: select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:19.842162 [ 2705 ] {02450_kill_distributed_query_deadlock_test_j5v4sbbn-50} TCPHandler: Code: 394. DB::Exception: Query was cancelled. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:28.757784 [ 3503 ] {} virtual void DB::SystemLog::savingThreadFunction() [LogElement = DB::ObjectStorageQueueLogElement]: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c45d4f6 8. ./contrib/llvm-project/libcxx/include/optional:260: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>) @ 0x000000002cbacf28 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1460: std::shared_ptr std::allocate_shared[abi:v15007], DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>, void>(std::allocator const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>&&) @ 0x000000002cbabafa 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::create(DB::StorageFactory::Arguments const&) @ 0x000000002cba21b8 11. ./build_docker/./src/Storages/StorageFactory.cpp:225: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x000000002b77bf53 12. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1659: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291b9254 13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/SystemLog.cpp:621: DB::SystemLog::prepareTable() @ 0x00000000298338b3 16. ./build_docker/./src/Interpreters/SystemLog.cpp:0: DB::SystemLog::savingThreadFunction() @ 0x0000000029836018 17. ./build_docker/./src/Common/SystemLogBase.cpp:0: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl::startup()::'lambda'()>(DB::SystemLogBase::startup()::'lambda'()&&)::'lambda'()::operator()() @ 0x000000001b01c408 18. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 20. asan_thread_start(void*) @ 0x000000000aa43059 21. ? @ 0x00007f12cd75eac3 22. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:08:29.094284 [ 3816 ] {89a0b212-9011-4c19-a835-9412d3b1bac0} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=6): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60386) (comment: 01701_if_tuple_segfault.sql) (in query: SELECT * FROM agg_table;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:0: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000027ab61b5 13. ./contrib/llvm-project/libcxx/include/vector:1647: DB::SerializationNamed::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000027b74737 14. ./build_docker/./src/DataTypes/Serializations/SerializationTuple.cpp:666: DB::SerializationTuple::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000027c2d1c6 15. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 16. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 19. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 22. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 23. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 24. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 25. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 26. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 27. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 28. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 29. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 31. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 2025.09.25 21:08:29.096583 [ 3816 ] {89a0b212-9011-4c19-a835-9412d3b1bac0} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=6): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:0: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000027ab61b5 13. ./contrib/llvm-project/libcxx/include/vector:1647: DB::SerializationNamed::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000027b74737 14. ./build_docker/./src/DataTypes/Serializations/SerializationTuple.cpp:666: DB::SerializationTuple::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000027c2d1c6 15. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 16. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 19. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 22. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 23. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 24. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 25. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 26. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 27. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 28. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 29. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 31. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 2025.09.25 21:08:30.154293 [ 2707 ] {e155fbba-a96d-4fa6-8ae9-51a939b2f491} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=7): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:44496) (comment: 00180_no_seek_avoiding_when_reading_from_cache.sh) (in query: -- AsynchronousReaderIgnoredBytes = 0: no seek-avoiding happened -- CachedReadBufferReadFromSourceBytes = 0: sanity check to ensure we read only from cache SELECT ProfileEvents['AsynchronousReaderIgnoredBytes'], ProfileEvents['CachedReadBufferReadFromSourceBytes'] FROM system.query_log WHERE query_id = '02906_read_from_cache_4502' AND type = 'QueryFinish' AND event_date >= yesterday() AND current_database = currentDatabase() ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:144: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002def1bcc 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 17. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 18. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 19. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 22. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 23. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 25. asan_thread_start(void*) @ 0x000000000aa43059 26. ? @ 0x00007f12cd75eac3 27. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:30.182266 [ 2707 ] {e155fbba-a96d-4fa6-8ae9-51a939b2f491} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=7): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:144: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002def1bcc 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 17. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 18. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 19. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 22. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 23. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 25. asan_thread_start(void*) @ 0x000000000aa43059 26. ? @ 0x00007f12cd75eac3 27. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:31.810793 [ 3656 ] {50ffeb7a-ed22-4f93-823e-e2ce535137fc} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60390) (comment: 00172_hits_joins.gen.sql) (in query: SELECT EventDate, hits, visits FROM ( SELECT EventDate, count() AS hits FROM test.hits GROUP BY EventDate ) ANY LEFT JOIN ( SELECT StartDate AS EventDate, sum(Sign) AS visits FROM test.visits GROUP BY EventDate ) USING EventDate ORDER BY hits DESC LIMIT 10 SETTINGS joined_subquery_requires_alias = 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 27. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 28. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:08:31.835925 [ 3656 ] {50ffeb7a-ed22-4f93-823e-e2ce535137fc} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 27. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 28. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:08:32.024573 [ 2705 ] {ae3770fa-3932-400c-afff-cc389ceee6b7} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=751, jobs=722). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60370) (comment: 00212_long_shard_aggregate_function_uniq.sql) (in query: SELECT Y, uniqCombined(12)(X) FROM (SELECT number AS X, (3*X*X - 7*X + 11) % 37 AS Y FROM system.numbers LIMIT 1000000) GROUP BY Y ORDER BY Y;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:32.048465 [ 2705 ] {ae3770fa-3932-400c-afff-cc389ceee6b7} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=751, jobs=722). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:34.294472 [ 3682 ] {} HTTPServerRequest: Got an HTTP request with no content length and no chunked/multipart encoding, it may be impossible to distinguish graceful EOF from abnormal connection loss 2025.09.25 21:08:35.131350 [ 3619 ] {2270ef27-41bb-402a-a9cb-04706bc80aba} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:44608) (comment: 00040_aggregating_materialized_view.sql) (in query: CREATE MATERIALIZED VIEW test.basic_00040 ENGINE = AggregatingMergeTree(StartDate, (CounterID, StartDate), 8192) POPULATE AS SELECT CounterID, StartDate, sumState(Sign) AS Visits, uniqState(UserID) AS Users FROM test.visits GROUP BY CounterID, StartDate;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 27. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 28. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:08:35.149510 [ 3619 ] {2270ef27-41bb-402a-a9cb-04706bc80aba} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 27. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 28. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:08:36.062268 [ 3851 ] {347aa760-157c-4fd8-a819-4e492778080e} executeQuery: Poco::Exception. Code: 1000, e.code() = 0, Bad URI syntax: URI contains invalid characters (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:44652) (comment: 00990_request_splitting.sql) (in query: SELECT * FROM url('http://127.0.0.1:1337/? HTTP/1.1\r\nTest: test', CSV, 'column1 String');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:179: Poco::URISyntaxException::URISyntaxException(String const&, int) @ 0x00000000343647ee 2. ./build_docker/./base/poco/Foundation/src/URI.cpp:743: Poco::URI::parse(String const&) @ 0x000000003443fdd0 3. ./build_docker/./base/poco/Foundation/src/URI.cpp:43: Poco::URI::URI(String const&, bool) @ 0x000000003443f5e3 4. ./build_docker/./src/Storages/StorageURL.cpp:0: DB::IStorageURLBase::IStorageURLBase(String const&, std::shared_ptr const&, DB::StorageID const&, String const&, std::optional const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, String const&, String const&, std::vector> const&, String const&, std::shared_ptr, bool) @ 0x000000002bd71acf 5. ./build_docker/./src/Storages/StorageURL.cpp:1366: DB::StorageURL::StorageURL(String const&, DB::StorageID const&, String const&, std::optional const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, String const&, std::shared_ptr const&, String const&, std::vector> const&, String const&, std::shared_ptr, bool) @ 0x000000002bd8414d 6. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr std::allocate_shared[abi:v15007], String const&, DB::StorageID, String const&, std::nullopt_t const&, DB::ColumnsDescription const&, DB::ConstraintsDescription, String, std::shared_ptr&, String const&, std::vector> const&, String const&, void>(std::allocator const&, String const&, DB::StorageID&&, String const&, std::nullopt_t const&, DB::ColumnsDescription const&, DB::ConstraintsDescription&&, String&&, std::shared_ptr&, String const&, std::vector> const&, String const&) @ 0x0000000025ff3925 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::TableFunctionURL::getStorage(String const&, String const&, DB::ColumnsDescription const&, std::shared_ptr, String const&, String const&) const @ 0x0000000025ff17db 8. ./build_docker/./src/TableFunctions/ITableFunctionFileLike.cpp:99: DB::ITableFunctionFileLike::executeImpl(std::shared_ptr const&, std::shared_ptr, String const&, DB::ColumnsDescription, bool) const @ 0x0000000025feb663 9. ./build_docker/./src/TableFunctions/ITableFunction.cpp:36: DB::ITableFunction::execute(std::shared_ptr const&, std::shared_ptr, String const&, DB::ColumnsDescription, bool, bool) const @ 0x00000000265c241e 10. ./build_docker/./src/Interpreters/Context.cpp:2234: DB::Context::executeTableFunction(std::shared_ptr const&, std::shared_ptr const&) @ 0x00000000285361d2 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4819: DB::QueryAnalyzer::resolveTableFunction(std::shared_ptr&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&, bool) @ 0x000000002837024e 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x00000000283ba404 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ad09 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:36.069752 [ 3851 ] {347aa760-157c-4fd8-a819-4e492778080e} TCPHandler: Code: 1000. DB::Exception: Bad URI syntax: URI contains invalid characters. (POCO_EXCEPTION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./base/poco/Foundation/src/Exception.cpp:179: Poco::URISyntaxException::URISyntaxException(String const&, int) @ 0x00000000343647ee 2. ./build_docker/./base/poco/Foundation/src/URI.cpp:743: Poco::URI::parse(String const&) @ 0x000000003443fdd0 3. ./build_docker/./base/poco/Foundation/src/URI.cpp:43: Poco::URI::URI(String const&, bool) @ 0x000000003443f5e3 4. ./build_docker/./src/Storages/StorageURL.cpp:0: DB::IStorageURLBase::IStorageURLBase(String const&, std::shared_ptr const&, DB::StorageID const&, String const&, std::optional const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, String const&, String const&, std::vector> const&, String const&, std::shared_ptr, bool) @ 0x000000002bd71acf 5. ./build_docker/./src/Storages/StorageURL.cpp:1366: DB::StorageURL::StorageURL(String const&, DB::StorageID const&, String const&, std::optional const&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, String const&, std::shared_ptr const&, String const&, std::vector> const&, String const&, std::shared_ptr, bool) @ 0x000000002bd8414d 6. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: std::shared_ptr std::allocate_shared[abi:v15007], String const&, DB::StorageID, String const&, std::nullopt_t const&, DB::ColumnsDescription const&, DB::ConstraintsDescription, String, std::shared_ptr&, String const&, std::vector> const&, String const&, void>(std::allocator const&, String const&, DB::StorageID&&, String const&, std::nullopt_t const&, DB::ColumnsDescription const&, DB::ConstraintsDescription&&, String&&, std::shared_ptr&, String const&, std::vector> const&, String const&) @ 0x0000000025ff3925 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::TableFunctionURL::getStorage(String const&, String const&, DB::ColumnsDescription const&, std::shared_ptr, String const&, String const&) const @ 0x0000000025ff17db 8. ./build_docker/./src/TableFunctions/ITableFunctionFileLike.cpp:99: DB::ITableFunctionFileLike::executeImpl(std::shared_ptr const&, std::shared_ptr, String const&, DB::ColumnsDescription, bool) const @ 0x0000000025feb663 9. ./build_docker/./src/TableFunctions/ITableFunction.cpp:36: DB::ITableFunction::execute(std::shared_ptr const&, std::shared_ptr, String const&, DB::ColumnsDescription, bool, bool) const @ 0x00000000265c241e 10. ./build_docker/./src/Interpreters/Context.cpp:2234: DB::Context::executeTableFunction(std::shared_ptr const&, std::shared_ptr const&) @ 0x00000000285361d2 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4819: DB::QueryAnalyzer::resolveTableFunction(std::shared_ptr&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&, bool) @ 0x000000002837024e 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x00000000283ba404 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ad09 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:39.694985 [ 3619 ] {44a02136-0c28-40f1-b9b4-55e9a87eb6e9} executeQuery: Code: 57. DB::Exception: Table test_f7mq5du4.t already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:40232) (comment: 00977_join_use_nulls_denny_crane.sql) (in query: create table t(a Int64, b Int64, c Nullable(String)) engine = Memory;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:39.718225 [ 3619 ] {44a02136-0c28-40f1-b9b4-55e9a87eb6e9} TCPHandler: Code: 57. DB::Exception: Table test_f7mq5du4.t already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:40.239254 [ 3078 ] {} DatabaseCatalog: Cannot drop tables. Will retry later.: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000286c6b9c 8. ./contrib/llvm-project/libcxx/include/vector:590: DB::DatabaseCatalog::dropTableDataTask() @ 0x00000000286c7466 9. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 11. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 14. asan_thread_start(void*) @ 0x000000000aa43059 15. ? @ 0x00007f12cd75eac3 16. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:08:45.222123 [ 2707 ] {b441b6ff-5750-4d84-bb1e-89b68e23833c} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=1): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:51340) (comment: 00046_uniq_upto_distributed.sql) (in query: SELECT `__table1`.`RegionID` AS `RegionID`, uniqExact(`__table1`.`UserID`) AS `u1`, uniqUpTo(10)(`__table1`.`UserID`) AS `u2` FROM `test`.`visits` AS `__table1` GROUP BY `__table1`.`RegionID` HAVING (uniqExact(`__table1`.`UserID`) <= 11) AND (uniqExact(`__table1`.`UserID`) != uniqUpTo(10)(`__table1`.`UserID`))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c5f2 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:144: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002def1bcc 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 22. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 23. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:08:45.271271 [ 2707 ] {b441b6ff-5750-4d84-bb1e-89b68e23833c} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=1): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c5f2 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:144: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002def1bcc 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 22. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 23. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:08:45.395364 [ 3656 ] {f42dd484-d221-4a34-b1c2-a1386373672f} executeQuery: Code: 439. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=1): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). Stack trace: 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c5f2 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:144: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002def1bcc 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 22. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 23. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 : While executing Remote. (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:40530) (comment: 00046_uniq_upto_distributed.sql) (in query: SELECT RegionID, uniqExact(UserID) AS u1, uniqUpTo(10)(UserID) AS u2 FROM remote('127.0.0.{1,2}', test, visits) GROUP BY RegionID HAVING u1 <= 11 AND u1 != u2), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001aeb974e 3. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000002cee6c6a 4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x000000002cef780e 5. ./build_docker/./src/Client/PacketReceiver.cpp:0: DB::PacketReceiver::Task::run(std::function, std::function) @ 0x000000002cf9767e 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278ab436 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278aaa44 8. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func>>(unsigned int, unsigned int) @ 0x00000000278a8394 2025.09.25 21:08:45.473932 [ 3656 ] {f42dd484-d221-4a34-b1c2-a1386373672f} TCPHandler: Code: 439. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=1): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). Stack trace: 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c5f2 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:144: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002def1bcc 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 22. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 23. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 : While executing Remote. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001aeb974e 3. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000002cee6c6a 4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x000000002cef780e 5. ./build_docker/./src/Client/PacketReceiver.cpp:0: DB::PacketReceiver::Task::run(std::function, std::function) @ 0x000000002cf9767e 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278ab436 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278aaa44 8. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func>>(unsigned int, unsigned int) @ 0x00000000278a8394 2025.09.25 21:08:45.576548 [ 3682 ] {c602176f-ba28-4bf2-96ac-98ea80758310} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:47698). (NETWORK_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:47698) (comment: 00046_uniq_upto_distributed.sql) (in query: SELECT `__table1`.`RegionID` AS `RegionID`, uniqExact(`__table1`.`UserID`) AS `u1`, uniqUpTo(10)(`__table1`.`UserID`) AS `u2` FROM `test`.`visits` AS `__table1` GROUP BY `__table1`.`RegionID` HAVING (uniqExact(`__table1`.`UserID`) <= 11) AND (uniqExact(`__table1`.`UserID`) != uniqUpTo(10)(`__table1`.`UserID`))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001b0bab02 4. ./build_docker/./src/IO/WriteBufferFromPocoSocketChunked.cpp:201: DB::WriteBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf09bf7 5. ./src/IO/WriteBuffer.h:66: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000002d190f6b 6. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18b989 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:45.709490 [ 3682 ] {c602176f-ba28-4bf2-96ac-98ea80758310} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:47698). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001b0bada2 4. ./build_docker/./src/IO/WriteBufferFromPocoSocketChunked.cpp:201: DB::WriteBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf09bf7 5. ./src/IO/WriteBuffer.h:66: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000002d1a9beb 6. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x000000002d18db59 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d176902 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:08:45.715836 [ 3682 ] {c602176f-ba28-4bf2-96ac-98ea80758310} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.1]:9000 -> [::ffff:127.0.0.1]:47698). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001b0bab02 4. ./build_docker/./src/IO/WriteBufferFromPocoSocketChunked.cpp:201: DB::WriteBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf09bf7 5. ./src/IO/WriteBuffer.h:66: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000002d190f6b 6. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18b989 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:45.726593 [ 3682 ] {c602176f-ba28-4bf2-96ac-98ea80758310} TCPHandler: Client has gone away. 2025.09.25 21:08:45.789392 [ 3816 ] {1e9ba14f-f499-4fb2-9b6e-bc367c9750e8} executeQuery: Code: 57. DB::Exception: Table test_13.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:40578) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:45.810627 [ 3816 ] {1e9ba14f-f499-4fb2-9b6e-bc367c9750e8} TCPHandler: Code: 57. DB::Exception: Table test_13.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:47.917434 [ 3682 ] {d61ea924-da29-41d3-afde-7b32ecf924dd} executeQuery: Code: 62. DB::Exception: Cannot TRUNCATE dictionary. (SYNTAX_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58034) (comment: 01037_polygon_dicts_correctness_all.sh) (in query: DROP DICTIONARY IF EXISTS dict_array;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:262: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f1adc 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:47.927979 [ 3682 ] {d61ea924-da29-41d3-afde-7b32ecf924dd} TCPHandler: Code: 62. DB::Exception: Cannot TRUNCATE dictionary. (SYNTAX_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:262: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f1adc 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:48.689865 [ 2707 ] {ec893659-69ab-439d-8bec-697283a13b0d} executeQuery: Code: 62. DB::Exception: Cannot TRUNCATE dictionary. (SYNTAX_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58070) (comment: 01037_polygon_dicts_correctness_all.sh) (in query: DROP DICTIONARY IF EXISTS dict_array;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:262: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f1adc 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:48.736794 [ 2707 ] {ec893659-69ab-439d-8bec-697283a13b0d} TCPHandler: Code: 62. DB::Exception: Cannot TRUNCATE dictionary. (SYNTAX_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:262: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f1adc 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:49.016095 [ 2707 ] {0e8416c4-5571-4594-98c5-5b5933ad4432} executeQuery: Code: 57. DB::Exception: Table test_13.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58096) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:49.042286 [ 2707 ] {0e8416c4-5571-4594-98c5-5b5933ad4432} TCPHandler: Code: 57. DB::Exception: Table test_13.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:50.002206 [ 2705 ] {f0e14c63-ba0a-497c-bea4-7659785a7c0c} executeQuery: Code: 48. DB::Exception: Truncate is not supported by storage Merge. (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58166) (comment: 02149_schema_inference_create_table_syntax.sh) (in query: drop table test_merge), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:50.014223 [ 2705 ] {f0e14c63-ba0a-497c-bea4-7659785a7c0c} TCPHandler: Code: 48. DB::Exception: Truncate is not supported by storage Merge. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:51.746818 [ 2707 ] {2b645040-bd3b-4189-926b-f81419fa909e} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'json' to function 'JSONArrayLength'. Expected: String, got: UInt8: In scope SELECT JSONArrayLength(2). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58116) (comment: 02568_json_array_length.sql) (in query: select JSONArrayLength(2);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception> const&, String, std::basic_string_view> const&, String>(int, FormatStringHelperImpl::type, std::type_identity> const&>::type, std::type_identity::type, std::type_identity> const&>::type, std::type_identity::type>, String&&, std::basic_string_view> const&, String&&, std::basic_string_view> const&, String&&) @ 0x000000000aaec6f5 4. DB::(anonymous namespace)::validateArgumentsImpl(DB::IFunction const&, std::vector> const&, unsigned long, std::vector> const&) @ 0x000000000aae6b1a 5. DB::validateFunctionArguments(DB::IFunction const&, std::vector> const&, std::vector> const&, std::vector> const&) @ 0x000000000aae5e2a 6. DB::(anonymous namespace)::FunctionJSONArrayLength::getReturnTypeImpl(std::vector> const&) const @ 0x000000000cce03d9 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:08:51.751299 [ 2707 ] {2b645040-bd3b-4189-926b-f81419fa909e} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'json' to function 'JSONArrayLength'. Expected: String, got: UInt8: In scope SELECT JSONArrayLength(2). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception> const&, String, std::basic_string_view> const&, String>(int, FormatStringHelperImpl::type, std::type_identity> const&>::type, std::type_identity::type, std::type_identity> const&>::type, std::type_identity::type>, String&&, std::basic_string_view> const&, String&&, std::basic_string_view> const&, String&&) @ 0x000000000aaec6f5 4. DB::(anonymous namespace)::validateArgumentsImpl(DB::IFunction const&, std::vector> const&, unsigned long, std::vector> const&) @ 0x000000000aae6b1a 5. DB::validateFunctionArguments(DB::IFunction const&, std::vector> const&, std::vector> const&, std::vector> const&) @ 0x000000000aae5e2a 6. DB::(anonymous namespace)::FunctionJSONArrayLength::getReturnTypeImpl(std::vector> const&) const @ 0x000000000cce03d9 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:08:51.843891 [ 2707 ] {467ad4c4-87d8-40c9-b818-57a1a87852ba} executeQuery: Code: 42. DB::Exception: Number of arguments for function JSONArrayLength doesn't match: passed 0, should be 1: In scope SELECT JSONArrayLength(). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58116) (comment: 02568_json_array_length.sql) (in query: select JSONArrayLength();), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, unsigned long&, unsigned long&) @ 0x000000000ccd84e0 4. DB::IFunctionOverloadResolver::checkNumberOfArguments(unsigned long) const @ 0x000000000ccd2a84 5. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd35cf 6. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 7. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 8. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 18. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 19. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 21. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 23. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 27. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007f12cd75eac3 30. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:51.869739 [ 2707 ] {467ad4c4-87d8-40c9-b818-57a1a87852ba} TCPHandler: Code: 42. DB::Exception: Number of arguments for function JSONArrayLength doesn't match: passed 0, should be 1: In scope SELECT JSONArrayLength(). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, unsigned long&, unsigned long&) @ 0x000000000ccd84e0 4. DB::IFunctionOverloadResolver::checkNumberOfArguments(unsigned long) const @ 0x000000000ccd2a84 5. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd35cf 6. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 7. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 8. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 18. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 19. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 21. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 23. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 27. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007f12cd75eac3 30. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:54.064652 [ 3682 ] {357ff5c8-9e64-44de-8e17-750512b9654a} executeQuery: Code: 107. DB::Exception: File /var/lib/clickhouse/user_files/nonexist.txt doesn't exist. (FILE_DOESNT_EXIST) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58352) (in query: select _path,_file from file('nonexist.txt', 'CSV', 'val1 char')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageFile.cpp:1604: DB::StorageFile::read(DB::QueryPlan&, std::vector> const&, std::shared_ptr const&, DB::SelectQueryInfo&, std::shared_ptr, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x000000002b79db62 5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:880: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr&, bool, bool) @ 0x0000000029469c4b 6. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1795: DB::buildJoinTreeQueryPlan(std::shared_ptr const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002945d99d 7. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000002942c986 8. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 9. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 13. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:54.070375 [ 3682 ] {357ff5c8-9e64-44de-8e17-750512b9654a} TCPHandler: Code: 107. DB::Exception: File /var/lib/clickhouse/user_files/nonexist.txt doesn't exist. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageFile.cpp:1604: DB::StorageFile::read(DB::QueryPlan&, std::vector> const&, std::shared_ptr const&, DB::SelectQueryInfo&, std::shared_ptr, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x000000002b79db62 5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:880: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr&, bool, bool) @ 0x0000000029469c4b 6. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1795: DB::buildJoinTreeQueryPlan(std::shared_ptr const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002945d99d 7. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000002942c986 8. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 9. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 13. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:54.506322 [ 2729 ] {} void DB::StorageBuffer::backgroundFlush(): Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002be5219f 8. ./build_docker/./src/Storages/StorageBuffer.cpp:1088: DB::StorageBuffer::backgroundFlush() @ 0x000000002be59b8f 9. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 11. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 14. asan_thread_start(void*) @ 0x000000000aa43059 15. ? @ 0x00007f12cd75eac3 16. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:08:54.909040 [ 3682 ] {d2b4bc7f-3224-4e86-8634-9897e636416a} executeQuery: Code: 57. DB::Exception: Table test_13.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58406) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:54.937485 [ 3682 ] {d2b4bc7f-3224-4e86-8634-9897e636416a} TCPHandler: Code: 57. DB::Exception: Table test_13.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:55.939903 [ 3682 ] {83dbc988-e587-43d8-957f-bab3336f1d86} executeQuery: Code: 48. DB::Exception: Truncate is not supported by storage Buffer. (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58432) (comment: 02149_schema_inference_create_table_syntax.sh) (in query: drop table test_buffer), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:55.942190 [ 3682 ] {83dbc988-e587-43d8-957f-bab3336f1d86} TCPHandler: Code: 48. DB::Exception: Truncate is not supported by storage Buffer. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:56.010198 [ 2705 ] {452988e6-79e8-44f2-96ca-2b27da1ec66d} executeQuery: Code: 48. DB::Exception: Truncate is not supported by storage Buffer. (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58286) (comment: 00061_storage_buffer.sql) (in query: DROP TABLE test.hits_buffer;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:56.025871 [ 2705 ] {452988e6-79e8-44f2-96ca-2b27da1ec66d} TCPHandler: Code: 48. DB::Exception: Truncate is not supported by storage Buffer. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:56.396935 [ 3627 ] {9ae62af4-2103-46b9-a638-557c8c6e3c10} executeQuery: Code: 48. DB::Exception: Truncate is not supported by storage Buffer. (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:59714) (comment: 02149_schema_inference_create_table_syntax.sh) (in query: DROP DATABASE IF EXISTS test_yrotdb1d ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:454: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f510c 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:56.404158 [ 3627 ] {9ae62af4-2103-46b9-a638-557c8c6e3c10} DynamicQueryHandler: Code: 48. DB::Exception: Truncate is not supported by storage Buffer. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:454: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f510c 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:08:57.689440 [ 3627 ] {5acde760-c863-44c2-824f-83121cabd6c0} executeQuery: Code: 57. DB::Exception: Table test_13.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:33102) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:57.705890 [ 3627 ] {5acde760-c863-44c2-824f-83121cabd6c0} TCPHandler: Code: 57. DB::Exception: Table test_13.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:58.055483 [ 2705 ] {329bd450-8d24-4a72-8518-4f2eb5a299f1} executeQuery: Code: 36. DB::Exception: Illegal (null) value column Nullable(UInt64) of argument of function range: In scope SELECT range(CAST(NULL, 'Nullable(UInt64)'), CAST('10', 'Nullable(UInt64)'), CAST('2', 'Nullable(UInt64)')). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58462) (comment: 02797_range_nullable.sql) (in query: SELECT range(Null::Nullable(UInt64), 10::Nullable(UInt64), 2::Nullable(UInt64));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::FunctionRange::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const::'lambda'(DB::ColumnWithTypeAndName const&)::operator()(DB::ColumnWithTypeAndName const&) const @ 0x000000001aca57f8 5. DB::FunctionRange::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000001ac9c923 6. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 7. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 9. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 10. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 11. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 12. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 23. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 28. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 2025.09.25 21:08:58.061223 [ 2705 ] {329bd450-8d24-4a72-8518-4f2eb5a299f1} TCPHandler: Code: 36. DB::Exception: Illegal (null) value column Nullable(UInt64) of argument of function range: In scope SELECT range(CAST(NULL, 'Nullable(UInt64)'), CAST('10', 'Nullable(UInt64)'), CAST('2', 'Nullable(UInt64)')). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::FunctionRange::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const::'lambda'(DB::ColumnWithTypeAndName const&)::operator()(DB::ColumnWithTypeAndName const&) const @ 0x000000001aca57f8 5. DB::FunctionRange::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000001ac9c923 6. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 7. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 9. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 10. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 11. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 12. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 23. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 28. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 2025.09.25 21:08:58.065226 [ 3619 ] {cf662680-60aa-4fd8-a18a-3a79ef85cb21} executeQuery: Code: 57. DB::Exception: Table test_dvtor41f.check_codec already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:33088) (comment: 01390_check_table_codec.sql) (in query: CREATE TABLE check_codec (`a` Int, `b` Int CODEC(Delta, ZSTD)) ENGINE = MergeTree ORDER BY a SETTINGS min_bytes_for_wide_part = '10M', index_granularity = 57363, ratio_of_defaults_for_sparse_serialization = 0., replace_long_file_name_to_hash = false, max_file_name_length = 0, min_bytes_for_full_part_storage = 0, compact_parts_max_bytes_to_buffer = 269114099, compact_parts_max_granules_to_buffer = 190, compact_parts_merge_max_bytes_to_prefetch_part = 11252766, merge_max_block_size = 14438, old_parts_lifetime = 480., prefer_fetch_merged_part_size_threshold = 10737418240, vertical_merge_algorithm_min_rows_to_activate = 1000000, vertical_merge_algorithm_min_columns_to_activate = 100, min_merge_bytes_to_use_direct_io = 2350195519, index_granularity_bytes = 4927788, concurrent_part_removal_threshold = 32, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = false, marks_compress_block_size = 25421, primary_key_compress_block_size = 31777), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:58.071258 [ 3619 ] {cf662680-60aa-4fd8-a18a-3a79ef85cb21} TCPHandler: Code: 57. DB::Exception: Table test_dvtor41f.check_codec already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:58.186819 [ 2705 ] {112ac22b-cd47-48e2-b089-9a98f7672bb4} executeQuery: Code: 36. DB::Exception: Illegal (null) value column Nullable(UInt64) of argument of function range: In scope SELECT range(CAST('0', 'Nullable(UInt64)'), CAST(NULL, 'Nullable(UInt64)'), CAST('2', 'Nullable(UInt64)')). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58462) (comment: 02797_range_nullable.sql) (in query: SELECT range(0::Nullable(UInt64), Null::Nullable(UInt64), 2::Nullable(UInt64));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::FunctionRange::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const::'lambda'(DB::ColumnWithTypeAndName const&)::operator()(DB::ColumnWithTypeAndName const&) const @ 0x000000001aca57f8 5. DB::FunctionRange::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000001ac9c923 6. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 7. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 9. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 10. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 11. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 12. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 23. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 28. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 2025.09.25 21:08:58.190662 [ 2705 ] {112ac22b-cd47-48e2-b089-9a98f7672bb4} TCPHandler: Code: 36. DB::Exception: Illegal (null) value column Nullable(UInt64) of argument of function range: In scope SELECT range(CAST('0', 'Nullable(UInt64)'), CAST(NULL, 'Nullable(UInt64)'), CAST('2', 'Nullable(UInt64)')). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::FunctionRange::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const::'lambda'(DB::ColumnWithTypeAndName const&)::operator()(DB::ColumnWithTypeAndName const&) const @ 0x000000001aca57f8 5. DB::FunctionRange::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000001ac9c923 6. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 7. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 9. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 10. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 11. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 12. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 23. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 28. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 2025.09.25 21:08:58.252900 [ 2705 ] {4bc5b303-829f-4953-aa6a-477b7ccc3f91} executeQuery: Code: 36. DB::Exception: Illegal (null) value column Nullable(UInt64) of argument of function range: In scope SELECT range(CAST('0', 'Nullable(UInt64)'), CAST('10', 'Nullable(UInt64)'), CAST(NULL, 'Nullable(UInt64)')). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58462) (comment: 02797_range_nullable.sql) (in query: SELECT range(0::Nullable(UInt64), 10::Nullable(UInt64), Null::Nullable(UInt64));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::FunctionRange::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const::'lambda'(DB::ColumnWithTypeAndName const&)::operator()(DB::ColumnWithTypeAndName const&) const @ 0x000000001aca57f8 5. DB::FunctionRange::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000001ac9c923 6. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 7. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 9. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 10. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 11. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 12. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 23. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 28. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 2025.09.25 21:08:58.256872 [ 2705 ] {4bc5b303-829f-4953-aa6a-477b7ccc3f91} TCPHandler: Code: 36. DB::Exception: Illegal (null) value column Nullable(UInt64) of argument of function range: In scope SELECT range(CAST('0', 'Nullable(UInt64)'), CAST('10', 'Nullable(UInt64)'), CAST(NULL, 'Nullable(UInt64)')). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::FunctionRange::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const::'lambda'(DB::ColumnWithTypeAndName const&)::operator()(DB::ColumnWithTypeAndName const&) const @ 0x000000001aca57f8 5. DB::FunctionRange::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000001ac9c923 6. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 7. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 9. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 10. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 11. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 12. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 23. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 28. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 2025.09.25 21:08:58.292853 [ 3619 ] {f75a5740-c3a0-4c80-a84f-af3dd879fc53} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34646) (comment: 01390_check_table_codec.sql) (in query: DROP DATABASE IF EXISTS test_dvtor41f ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000291f4e6a 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 13. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 14. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 15. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 19. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 20. asan_thread_start(void*) @ 0x000000000aa43059 21. ? @ 0x00007f12cd75eac3 22. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:58.297746 [ 3619 ] {f75a5740-c3a0-4c80-a84f-af3dd879fc53} DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000291f4e6a 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 13. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 14. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 15. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 19. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 20. asan_thread_start(void*) @ 0x000000000aa43059 21. ? @ 0x00007f12cd75eac3 22. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:08:58.743121 [ 2705 ] {29356b53-82a3-4a85-8bd1-76481ee4fdf9} executeQuery: Code: 36. DB::Exception: Illegal (null) value column Const(Nullable(UInt8)) of argument of function range. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58462) (comment: 02797_range_nullable.sql) (in query: SELECT range(Null::Nullable(UInt8), materialize(1));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::FunctionRange::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const::'lambda'(DB::ColumnWithTypeAndName const&)::operator()(DB::ColumnWithTypeAndName const&) const @ 0x000000001aca57f8 5. DB::FunctionRange::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000001ac9c923 6. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 7. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 9. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 10. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 11. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::ActionsDAG::evaluatePartialResult(std::unordered_map, std::equal_to, std::allocator>>&, std::vector> const&, unsigned long, bool) @ 0x0000000028259428 12. ./contrib/llvm-project/libcxx/include/vector:961: DB::ActionsDAG::updateHeader(DB::Block const&) const @ 0x0000000028256e17 13. ./build_docker/./src/Processors/Transforms/ExpressionTransform.cpp:10: DB::ExpressionTransform::transformHeader(DB::Block const&, DB::ActionsDAG const&) @ 0x000000002d9c2197 14. ./build_docker/./src/Processors/QueryPlan/ExpressionStep.cpp:0: DB::ExpressionStep::ExpressionStep(DB::DataStream const&, DB::ActionsDAG) @ 0x000000002dd8d279 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::(anonymous namespace)::addExpressionStep(DB::QueryPlan&, std::shared_ptr&, String const&, std::unordered_set, std::hash>, std::equal_to>, std::allocator>>&) @ 0x000000002943bd73 16. ./contrib/llvm-project/libcxx/include/string:1499: DB::Planner::buildPlanForQueryNode() @ 0x0000000029434460 17. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 18. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 19. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 20. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 22. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 26. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 27. asan_thread_start(void*) @ 0x000000000aa43059 28. ? @ 0x00007f12cd75eac3 29. ? @ 0x00007f12cd7f0850 2025.09.25 21:08:58.748936 [ 2705 ] {29356b53-82a3-4a85-8bd1-76481ee4fdf9} TCPHandler: Code: 36. DB::Exception: Illegal (null) value column Const(Nullable(UInt8)) of argument of function range. (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::FunctionRange::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const::'lambda'(DB::ColumnWithTypeAndName const&)::operator()(DB::ColumnWithTypeAndName const&) const @ 0x000000001aca57f8 5. DB::FunctionRange::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000001ac9c923 6. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 7. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 9. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 10. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 11. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::ActionsDAG::evaluatePartialResult(std::unordered_map, std::equal_to, std::allocator>>&, std::vector> const&, unsigned long, bool) @ 0x0000000028259428 12. ./contrib/llvm-project/libcxx/include/vector:961: DB::ActionsDAG::updateHeader(DB::Block const&) const @ 0x0000000028256e17 13. ./build_docker/./src/Processors/Transforms/ExpressionTransform.cpp:10: DB::ExpressionTransform::transformHeader(DB::Block const&, DB::ActionsDAG const&) @ 0x000000002d9c2197 14. ./build_docker/./src/Processors/QueryPlan/ExpressionStep.cpp:0: DB::ExpressionStep::ExpressionStep(DB::DataStream const&, DB::ActionsDAG) @ 0x000000002dd8d279 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::(anonymous namespace)::addExpressionStep(DB::QueryPlan&, std::shared_ptr&, String const&, std::unordered_set, std::hash>, std::equal_to>, std::allocator>>&) @ 0x000000002943bd73 16. ./contrib/llvm-project/libcxx/include/string:1499: DB::Planner::buildPlanForQueryNode() @ 0x0000000029434460 17. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 18. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 19. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 20. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 22. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 26. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 27. asan_thread_start(void*) @ 0x000000000aa43059 28. ? @ 0x00007f12cd75eac3 29. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:00.912253 [ 3627 ] {e4437503-6242-4f44-932c-6dd4dfcb13da} executeQuery: Code: 57. DB::Exception: Table test_13.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:33234) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:00.926372 [ 3627 ] {e4437503-6242-4f44-932c-6dd4dfcb13da} TCPHandler: Code: 57. DB::Exception: Table test_13.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:01.475719 [ 3627 ] {992c3214-b329-450c-a346-c62be744e2a3} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=751, jobs=722). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:33262) (comment: 02493_analyzer_uniq_injective_functions_elimination.sql) (in query: -- Test with chain of injective functions EXPLAIN QUERY TREE SELECT uniqCombined(tuple(materialize(tuple(number)))) FROM numbers(10);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:01.481628 [ 3627 ] {992c3214-b329-450c-a346-c62be744e2a3} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=751, jobs=722). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:08.765932 [ 3816 ] {87844cc9-46bc-43f4-a75f-ef7cfd2948c0} executeQuery: Code: 184. DB::Exception: Aggregate function max(x1) is found in GROUP BY in query. (ILLEGAL_AGGREGATION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:33312) (comment: 02006_test_positional_arguments.sql) (in query: select max(x1), x2 from test group by 1, 2;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&) @ 0x000000000e9cb0ba 4. ./build_docker/./src/Analyzer/AggregationUtils.cpp:43: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424cba 5. ./src/Analyzer/InDepthQueryTreeVisitor.h:0: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424b15 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::assertNoAggregateFunctionNodes(std::shared_ptr const&, String const&) @ 0x000000002842547b 7. ./contrib/llvm-project/libcxx/include/string:1499: DB::validateAggregates(std::shared_ptr const&, DB::AggregatesValidationParams) @ 0x000000002841bd4c 8. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835c627 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 11. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 14. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 15. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 17. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 19. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 23. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 24. asan_thread_start(void*) @ 0x000000000aa43059 25. ? @ 0x00007f12cd75eac3 26. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:08.779692 [ 3816 ] {87844cc9-46bc-43f4-a75f-ef7cfd2948c0} TCPHandler: Code: 184. DB::Exception: Aggregate function max(x1) is found in GROUP BY in query. (ILLEGAL_AGGREGATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&) @ 0x000000000e9cb0ba 4. ./build_docker/./src/Analyzer/AggregationUtils.cpp:43: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424cba 5. ./src/Analyzer/InDepthQueryTreeVisitor.h:0: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424b15 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::assertNoAggregateFunctionNodes(std::shared_ptr const&, String const&) @ 0x000000002842547b 7. ./contrib/llvm-project/libcxx/include/string:1499: DB::validateAggregates(std::shared_ptr const&, DB::AggregatesValidationParams) @ 0x000000002841bd4c 8. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835c627 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 11. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 14. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 15. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 17. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 19. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 23. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 24. asan_thread_start(void*) @ 0x000000000aa43059 25. ? @ 0x00007f12cd75eac3 26. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:08.981700 [ 3816 ] {060a009f-6668-4715-ae05-c56d3201a389} executeQuery: Code: 184. DB::Exception: Aggregate function max(x1) is found in GROUP BY in query. (ILLEGAL_AGGREGATION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:33312) (comment: 02006_test_positional_arguments.sql) (in query: select 1 + max(x1), x2 from test group by 1, 2;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&) @ 0x000000000e9cb0ba 4. ./build_docker/./src/Analyzer/AggregationUtils.cpp:43: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424cba 5. ./src/Analyzer/InDepthQueryTreeVisitor.h:0: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424b15 6. ./src/Analyzer/InDepthQueryTreeVisitor.h:0: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424b15 7. ./src/Analyzer/InDepthQueryTreeVisitor.h:0: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424b15 8. ./contrib/llvm-project/libcxx/include/string:1499: DB::assertNoAggregateFunctionNodes(std::shared_ptr const&, String const&) @ 0x000000002842547b 9. ./contrib/llvm-project/libcxx/include/string:1499: DB::validateAggregates(std::shared_ptr const&, DB::AggregatesValidationParams) @ 0x000000002841bd4c 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835c627 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 13. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 15. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 16. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 17. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 19. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 21. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 22. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 24. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 25. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:08.985973 [ 3816 ] {060a009f-6668-4715-ae05-c56d3201a389} TCPHandler: Code: 184. DB::Exception: Aggregate function max(x1) is found in GROUP BY in query. (ILLEGAL_AGGREGATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&) @ 0x000000000e9cb0ba 4. ./build_docker/./src/Analyzer/AggregationUtils.cpp:43: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424cba 5. ./src/Analyzer/InDepthQueryTreeVisitor.h:0: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424b15 6. ./src/Analyzer/InDepthQueryTreeVisitor.h:0: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424b15 7. ./src/Analyzer/InDepthQueryTreeVisitor.h:0: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424b15 8. ./contrib/llvm-project/libcxx/include/string:1499: DB::assertNoAggregateFunctionNodes(std::shared_ptr const&, String const&) @ 0x000000002842547b 9. ./contrib/llvm-project/libcxx/include/string:1499: DB::validateAggregates(std::shared_ptr const&, DB::AggregatesValidationParams) @ 0x000000002841bd4c 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835c627 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 13. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 15. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 16. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 17. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 19. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 21. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 22. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 24. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 25. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:09.066031 [ 3816 ] {60dc0330-0ab4-404d-b183-779893f4b75c} executeQuery: Code: 184. DB::Exception: Aggregate function max(x1) is found in GROUP BY in query. (ILLEGAL_AGGREGATION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:33312) (comment: 02006_test_positional_arguments.sql) (in query: select max(x1), x2 from test group by -2, -1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&) @ 0x000000000e9cb0ba 4. ./build_docker/./src/Analyzer/AggregationUtils.cpp:43: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424cba 5. ./src/Analyzer/InDepthQueryTreeVisitor.h:0: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424b15 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::assertNoAggregateFunctionNodes(std::shared_ptr const&, String const&) @ 0x000000002842547b 7. ./contrib/llvm-project/libcxx/include/string:1499: DB::validateAggregates(std::shared_ptr const&, DB::AggregatesValidationParams) @ 0x000000002841bd4c 8. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835c627 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 11. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 14. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 15. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 17. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 19. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 23. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 24. asan_thread_start(void*) @ 0x000000000aa43059 25. ? @ 0x00007f12cd75eac3 26. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:09.070222 [ 3816 ] {60dc0330-0ab4-404d-b183-779893f4b75c} TCPHandler: Code: 184. DB::Exception: Aggregate function max(x1) is found in GROUP BY in query. (ILLEGAL_AGGREGATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&) @ 0x000000000e9cb0ba 4. ./build_docker/./src/Analyzer/AggregationUtils.cpp:43: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424cba 5. ./src/Analyzer/InDepthQueryTreeVisitor.h:0: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424b15 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::assertNoAggregateFunctionNodes(std::shared_ptr const&, String const&) @ 0x000000002842547b 7. ./contrib/llvm-project/libcxx/include/string:1499: DB::validateAggregates(std::shared_ptr const&, DB::AggregatesValidationParams) @ 0x000000002841bd4c 8. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835c627 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 11. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 14. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 15. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 17. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 19. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 23. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 24. asan_thread_start(void*) @ 0x000000000aa43059 25. ? @ 0x00007f12cd75eac3 26. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:09.098272 [ 3682 ] {f4e22093-60c5-4b33-8a99-90665bd932b8} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:33362) (comment: 02462_distributions.sql) (in query: # Mean should be around 0 SELECT DISTINCT if (m >= toFloat64(-0.2) AND m <= toFloat64(0.2), 'Ok', 'Fail') FROM (SELECT avg(a) as m FROM (SELECT randNormal(0, 5) AS a FROM numbers(100000)));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:306: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349fbb 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:09.101589 [ 3682 ] {f4e22093-60c5-4b33-8a99-90665bd932b8} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=10, jobs=10). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:306: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349fbb 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:09.147140 [ 3816 ] {28ea4f85-f26f-499d-9e12-1f1f16a709bb} executeQuery: Code: 184. DB::Exception: Aggregate function max(x1) is found in GROUP BY in query. (ILLEGAL_AGGREGATION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:33312) (comment: 02006_test_positional_arguments.sql) (in query: select 1 + max(x1), x2 from test group by -2, -1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&) @ 0x000000000e9cb0ba 4. ./build_docker/./src/Analyzer/AggregationUtils.cpp:43: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424cba 5. ./src/Analyzer/InDepthQueryTreeVisitor.h:0: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424b15 6. ./src/Analyzer/InDepthQueryTreeVisitor.h:0: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424b15 7. ./src/Analyzer/InDepthQueryTreeVisitor.h:0: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424b15 8. ./contrib/llvm-project/libcxx/include/string:1499: DB::assertNoAggregateFunctionNodes(std::shared_ptr const&, String const&) @ 0x000000002842547b 9. ./contrib/llvm-project/libcxx/include/string:1499: DB::validateAggregates(std::shared_ptr const&, DB::AggregatesValidationParams) @ 0x000000002841bd4c 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835c627 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 13. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 15. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 16. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 17. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 19. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 21. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 22. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 24. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 25. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:09.149894 [ 3816 ] {28ea4f85-f26f-499d-9e12-1f1f16a709bb} TCPHandler: Code: 184. DB::Exception: Aggregate function max(x1) is found in GROUP BY in query. (ILLEGAL_AGGREGATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&) @ 0x000000000e9cb0ba 4. ./build_docker/./src/Analyzer/AggregationUtils.cpp:43: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424cba 5. ./src/Analyzer/InDepthQueryTreeVisitor.h:0: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424b15 6. ./src/Analyzer/InDepthQueryTreeVisitor.h:0: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424b15 7. ./src/Analyzer/InDepthQueryTreeVisitor.h:0: DB::InDepthQueryTreeVisitor::visit(std::shared_ptr const&) @ 0x0000000028424b15 8. ./contrib/llvm-project/libcxx/include/string:1499: DB::assertNoAggregateFunctionNodes(std::shared_ptr const&, String const&) @ 0x000000002842547b 9. ./contrib/llvm-project/libcxx/include/string:1499: DB::validateAggregates(std::shared_ptr const&, DB::AggregatesValidationParams) @ 0x000000002841bd4c 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835c627 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 13. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 15. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 16. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 17. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 19. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 21. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 22. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 24. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 25. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:09.400660 [ 3816 ] {b359bfdd-106e-4657-8746-7fdca2fd0343} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=751, jobs=722). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:33312) (comment: 02006_test_positional_arguments.sql) (in query: explain syntax select x1 + x3, x3 from test group by -2, -1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:09.415253 [ 3816 ] {b359bfdd-106e-4657-8746-7fdca2fd0343} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=751, jobs=722). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:13.497209 [ 3656 ] {475d19b7-ba60-4f94-9068-ca9c280b9d8c} executeQuery: Code: 407. DB::Exception: Numeric overflow: while executing 'FUNCTION plus(toDateTime(toStartOfFifteenMinutes(plus(_CAST(1000_DateTime, 'DateTime'_String), multiply(__table2.number, 65536_UInt32)))) :: 3, multiply(__table2.number, 9223372036854775807_UInt64) :: 2) -> plus(toDateTime(toStartOfFifteenMinutes(plus(_CAST(1000_DateTime, 'DateTime'_String), multiply(__table2.number, 65536_UInt32)))), multiply(__table2.number, 9223372036854775807_UInt64)) DateTime : 0'. (DECIMAL_OVERFLOW) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53674) (comment: 02772_jit_date_time_add.sql) (in query: SELECT DISTINCT result FROM (SELECT toStartOfFifteenMinutes(toDateTime(toStartOfFifteenMinutes(toDateTime(1000.0001220703125) + (number * 65536))) + (number * 9223372036854775807)) AS result FROM system.numbers LIMIT 1048576) ORDER BY result DESC NULLS FIRST FORMAT Null;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. void DB::Processor::vectorVector, DB::ColumnVector, DB::ColumnVector>(DB::ColumnVector const&, DB::ColumnVector&, DB::ColumnVector const&, DateLUTImpl const&, unsigned short, unsigned long) const @ 0x000000000cee308f 5. void DB::Processor::vectorVector, DB::ColumnVector>(DB::ColumnVector const&, DB::ColumnVector&, DB::IColumn const&, DateLUTImpl const&, unsigned short, unsigned long) const @ 0x000000000cee06b6 6. DB::DateTimeAddIntervalImpl::execute(DB::AddSecondsImpl, std::vector> const&, std::shared_ptr const&, unsigned short, unsigned long) @ 0x000000000cecf7ae 7. DB::FunctionDateOrDateTimeAddInterval::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000cecc3a7 8. DB::FunctionToExecutableFunctionAdaptor::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac9135 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccce5b 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. DB::IFunctionBase::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000aac7d8f 13. DB::FunctionBinaryArithmetic::executeDateTimeIntervalPlusMinus(std::vector> const&, std::shared_ptr const&, unsigned long, std::shared_ptr const&) const @ 0x0000000016eac11f 14. DB::FunctionBinaryArithmetic::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016eaa360 15. DB::FunctionToExecutableFunctionAdaptor::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac9135 16. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccce5b 17. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 18. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 19. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::ExpressionActions::execute(DB::Block&, unsigned long&, bool, bool) const @ 0x00000000286fc2bd 20. ./build_docker/./src/Processors/Transforms/ExpressionTransform.cpp:0: DB::ExpressionTransform::transform(DB::Chunk&) @ 0x000000002d9c29ae 21. ./contrib/llvm-project/libcxx/include/__utility/swap.h:35: DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x000000001b49a31f 22. ./build_docker/./src/Processors/ISimpleTransform.cpp:99: DB::ISimpleTransform::work() @ 0x000000002d323859 23. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 27. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:09:13.584524 [ 3656 ] {475d19b7-ba60-4f94-9068-ca9c280b9d8c} TCPHandler: Code: 407. DB::Exception: Numeric overflow: while executing 'FUNCTION plus(toDateTime(toStartOfFifteenMinutes(plus(_CAST(1000_DateTime, 'DateTime'_String), multiply(__table2.number, 65536_UInt32)))) :: 3, multiply(__table2.number, 9223372036854775807_UInt64) :: 2) -> plus(toDateTime(toStartOfFifteenMinutes(plus(_CAST(1000_DateTime, 'DateTime'_String), multiply(__table2.number, 65536_UInt32)))), multiply(__table2.number, 9223372036854775807_UInt64)) DateTime : 0'. (DECIMAL_OVERFLOW), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. void DB::Processor::vectorVector, DB::ColumnVector, DB::ColumnVector>(DB::ColumnVector const&, DB::ColumnVector&, DB::ColumnVector const&, DateLUTImpl const&, unsigned short, unsigned long) const @ 0x000000000cee308f 5. void DB::Processor::vectorVector, DB::ColumnVector>(DB::ColumnVector const&, DB::ColumnVector&, DB::IColumn const&, DateLUTImpl const&, unsigned short, unsigned long) const @ 0x000000000cee06b6 6. DB::DateTimeAddIntervalImpl::execute(DB::AddSecondsImpl, std::vector> const&, std::shared_ptr const&, unsigned short, unsigned long) @ 0x000000000cecf7ae 7. DB::FunctionDateOrDateTimeAddInterval::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000cecc3a7 8. DB::FunctionToExecutableFunctionAdaptor::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac9135 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccce5b 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. DB::IFunctionBase::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000aac7d8f 13. DB::FunctionBinaryArithmetic::executeDateTimeIntervalPlusMinus(std::vector> const&, std::shared_ptr const&, unsigned long, std::shared_ptr const&) const @ 0x0000000016eac11f 14. DB::FunctionBinaryArithmetic::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016eaa360 15. DB::FunctionToExecutableFunctionAdaptor::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac9135 16. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccce5b 17. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 18. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 19. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::ExpressionActions::execute(DB::Block&, unsigned long&, bool, bool) const @ 0x00000000286fc2bd 20. ./build_docker/./src/Processors/Transforms/ExpressionTransform.cpp:0: DB::ExpressionTransform::transform(DB::Chunk&) @ 0x000000002d9c29ae 21. ./contrib/llvm-project/libcxx/include/__utility/swap.h:35: DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x000000001b49a31f 22. ./build_docker/./src/Processors/ISimpleTransform.cpp:99: DB::ISimpleTransform::work() @ 0x000000002d323859 23. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 27. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:09:14.064220 [ 2705 ] {d3a7aa70-1ddc-4acd-8bdc-3efd313756f2} executeQuery: Code: 48. DB::Exception: Truncate is not supported by storage Merge. (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53616) (comment: 01298_alter_merge.sql) (in query: drop table merge;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:14.090340 [ 2705 ] {d3a7aa70-1ddc-4acd-8bdc-3efd313756f2} TCPHandler: Code: 48. DB::Exception: Truncate is not supported by storage Merge. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:20.688226 [ 2707 ] {32990a02-dd78-48bf-96d5-46fafb00b9b2} executeQuery: Code: 57. DB::Exception: Table test_3.test already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:54564) (comment: 03261_variant_permutation_bug.sql) (in query: create table test (x UInt64, d Variant(UInt64)) engine=Memory;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:20.714456 [ 2707 ] {32990a02-dd78-48bf-96d5-46fafb00b9b2} TCPHandler: Code: 57. DB::Exception: Table test_3.test already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:21.911343 [ 3355 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {df97ab97-a5cf-4ea8-bd10-026470f5fb5c::202509_1_26_5}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:24.798241 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:24.834447 [ 3619 ] {ba20f97f-db83-46d6-8b40-5566121d24b1} executeQuery: Code: 341. DB::Exception: Exception happened during execution of mutation 'mutation_4.txt' with part 'all_1_2_1_3' reason: 'Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:54594) (comment: 02521_lightweight_delete_and_ttl.sql) (in query: ALTER TABLE lwd_test_02521 MODIFY TTL event_time + INTERVAL 1 MONTH SETTINGS mutations_sync = 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, char const*&&, String&&, String&, String&) @ 0x000000002c76d4c6 4. ./build_docker/./src/Storages/MergeTree/MergeTreeMutationStatus.cpp:26: DB::checkMutationStatus(std::optional&, std::set, std::allocator> const&) @ 0x000000002c76d210 5. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::waitForMutation(long, String const&, bool) @ 0x000000002cbbc805 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::StorageMergeTree::alter(DB::AlterCommands const&, std::shared_ptr, std::unique_lock&) @ 0x000000002cbb62e3 7. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:210: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000029174586 8. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:24.847461 [ 3619 ] {ba20f97f-db83-46d6-8b40-5566121d24b1} TCPHandler: Code: 341. DB::Exception: Exception happened during execution of mutation 'mutation_4.txt' with part 'all_1_2_1_3' reason: 'Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, char const*&&, String&&, String&, String&) @ 0x000000002c76d4c6 4. ./build_docker/./src/Storages/MergeTree/MergeTreeMutationStatus.cpp:26: DB::checkMutationStatus(std::optional&, std::set, std::allocator> const&) @ 0x000000002c76d210 5. ./build_docker/./src/Storages/StorageMergeTree.cpp:0: DB::StorageMergeTree::waitForMutation(long, String const&, bool) @ 0x000000002cbbc805 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::StorageMergeTree::alter(DB::AlterCommands const&, std::shared_ptr, std::unique_lock&) @ 0x000000002cbb62e3 7. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:210: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000029174586 8. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:24.961611 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:25.620421 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:25.644309 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:25.885751 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:25.914382 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:26.234940 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:26.283254 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:26.751566 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:26.775985 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:27.378812 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:27.412413 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:28.357002 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:28.580195 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:29.065747 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:29.096892 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:30.328608 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:30.347168 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:30.833892 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:30.876385 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:31.543402 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:31.569481 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:32.820064 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:32.863339 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:34.021789 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:34.058563 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:34.976902 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:35.078511 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:35.723511 [ 3358 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {68caad89-531d-40c0-87c3-4b3574cf49ff::202509_1_10_2}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=11, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:35.752673 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:35.777330 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:37.014985 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:37.029909 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:37.367679 [ 7168 ] {1be86209-35a3-4895-90be-34ee23db0c56} executeQuery: Code: 48. DB::Exception: Can't execute any of specified algorithms for specified strictness/kind and right storage type. (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50562) (comment: 00063_loyalty_joins.sql) (in query: SELECT loyalty, count() FROM test.hits ANY LEFT JOIN ( SELECT UserID, sum(SearchEngineID = 2) AS yandex, sum(SearchEngineID = 3) AS google, toInt8(if(yandex > google, yandex / (yandex + google), -google / (yandex + google)) * 10) AS loyalty FROM test.hits WHERE (SearchEngineID = 2) OR (SearchEngineID = 3) GROUP BY UserID HAVING (yandex + google) > 10 ) USING UserID GROUP BY loyalty ORDER BY loyalty ASC;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Planner/PlannerJoins.cpp:917: DB::chooseJoinAlgorithm(std::shared_ptr&, std::shared_ptr const&, DB::Block const&, DB::Block const&, std::shared_ptr const&) @ 0x00000000294a6d9f 5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1490: DB::(anonymous namespace)::buildQueryPlanForJoinNode(std::shared_ptr const&, DB::JoinTreeQueryPlan, DB::JoinTreeQueryPlan, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002947642f 6. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::buildJoinTreeQueryPlan(std::shared_ptr const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002945f174 7. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000002942c986 8. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 9. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 13. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:37.372862 [ 7168 ] {1be86209-35a3-4895-90be-34ee23db0c56} TCPHandler: Code: 48. DB::Exception: Can't execute any of specified algorithms for specified strictness/kind and right storage type. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Planner/PlannerJoins.cpp:917: DB::chooseJoinAlgorithm(std::shared_ptr&, std::shared_ptr const&, DB::Block const&, DB::Block const&, std::shared_ptr const&) @ 0x00000000294a6d9f 5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1490: DB::(anonymous namespace)::buildQueryPlanForJoinNode(std::shared_ptr const&, DB::JoinTreeQueryPlan, DB::JoinTreeQueryPlan, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002947642f 6. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::buildJoinTreeQueryPlan(std::shared_ptr const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002945f174 7. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000002942c986 8. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 9. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 13. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:37.873115 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:37.897699 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:38.162493 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:38.196080 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:39.213542 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:39.237254 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:39.453577 [ 7168 ] {4fb3f149-d601-4336-b3d3-254452deeee5} executeQuery: Code: 35. DB::Exception: Number of arguments for function parseTimeDelta doesn't match: passed 0, should be 1.: In scope SELECT parseTimeDelta(). (TOO_FEW_ARGUMENTS_FOR_FUNCTION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50566) (comment: 02354_parse_timedelta.sql) (in query: -- invalid expressions SELECT parseTimeDelta();), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, unsigned long&&) @ 0x000000000aaadc3a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x0000000016c17fba 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:39.458412 [ 7168 ] {4fb3f149-d601-4336-b3d3-254452deeee5} TCPHandler: Code: 35. DB::Exception: Number of arguments for function parseTimeDelta doesn't match: passed 0, should be 1.: In scope SELECT parseTimeDelta(). (TOO_FEW_ARGUMENTS_FOR_FUNCTION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, unsigned long&&) @ 0x000000000aaadc3a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x0000000016c17fba 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:39.682811 [ 7168 ] {eeff1364-d688-4678-b44a-071ffabb397d} executeQuery: Code: 34. DB::Exception: Number of arguments for function parseTimeDelta doesn't match: passed 2, should be 1.: In scope SELECT parseTimeDelta('1yr', 1). (TOO_MANY_ARGUMENTS_FOR_FUNCTION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50566) (comment: 02354_parse_timedelta.sql) (in query: SELECT parseTimeDelta('1yr', 1);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, unsigned long&&) @ 0x000000000aaadc3a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x0000000016c180de 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:39.685729 [ 7168 ] {eeff1364-d688-4678-b44a-071ffabb397d} TCPHandler: Code: 34. DB::Exception: Number of arguments for function parseTimeDelta doesn't match: passed 2, should be 1.: In scope SELECT parseTimeDelta('1yr', 1). (TOO_MANY_ARGUMENTS_FOR_FUNCTION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, unsigned long&&) @ 0x000000000aaadc3a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x0000000016c180de 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:39.890795 [ 7168 ] {55119455-9456-4137-a78b-d5cf90226150} executeQuery: Code: 43. DB::Exception: Cannot format UInt8 as time string.: In scope SELECT parseTimeDelta(1). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50566) (comment: 02354_parse_timedelta.sql) (in query: SELECT parseTimeDelta(1);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. DB::(anonymous namespace)::FunctionParseTimeDelta::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x0000000016c1821e 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:39.894329 [ 7168 ] {55119455-9456-4137-a78b-d5cf90226150} TCPHandler: Code: 43. DB::Exception: Cannot format UInt8 as time string.: In scope SELECT parseTimeDelta(1). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. DB::(anonymous namespace)::FunctionParseTimeDelta::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x0000000016c1821e 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:40.050810 [ 7168 ] {7333d9d1-5b48-442f-bf88-edb8a7720a24} executeQuery: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, don't find valid characters, str: " ".: In scope SELECT parseTimeDelta(' '). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50566) (comment: 02354_parse_timedelta.sql) (in query: SELECT parseTimeDelta(' ');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c16c88 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:40.054813 [ 7168 ] {7333d9d1-5b48-442f-bf88-edb8a7720a24} TCPHandler: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, don't find valid characters, str: " ".: In scope SELECT parseTimeDelta(' '). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c16c88 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:40.261002 [ 7178 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::ffff:127.0.0.1]:59080, local: [::ffff:127.0.0.1]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./src/IO/ReadBuffer.h:0: void DB::varint_impl::readVarUInt(unsigned long&, DB::ReadBuffer&) @ 0x000000001aea644a 8. ./src/IO/ReadHelpers.h:143: DB::readStringBinary(String&, DB::ReadBuffer&, unsigned long) @ 0x000000001aef6572 9. ./build_docker/./src/Server/TCPHandler.cpp:1653: DB::TCPHandler::receiveAddendum() @ 0x000000002d1869eb 10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d174260 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:40.313705 [ 7168 ] {39772978-3e1b-4f5e-b5d4-25aea6a2831e} executeQuery: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, find number failed, str: "-1yr".: In scope SELECT parseTimeDelta('-1yr'). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50566) (comment: 02354_parse_timedelta.sql) (in query: SELECT parseTimeDelta('-1yr');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c1700f 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:40.321232 [ 7168 ] {39772978-3e1b-4f5e-b5d4-25aea6a2831e} TCPHandler: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, find number failed, str: "-1yr".: In scope SELECT parseTimeDelta('-1yr'). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c1700f 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:40.349489 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:40.375784 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:40.620069 [ 7168 ] {fe2ff5aa-6861-4f14-824d-01b4ff1ce5ee} executeQuery: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, str: "1yr-".: In scope SELECT parseTimeDelta('1yr-'). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50566) (comment: 02354_parse_timedelta.sql) (in query: SELECT parseTimeDelta('1yr-');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c17250 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:40.624017 [ 7168 ] {fe2ff5aa-6861-4f14-824d-01b4ff1ce5ee} TCPHandler: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, str: "1yr-".: In scope SELECT parseTimeDelta('1yr-'). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c17250 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:40.677390 [ 7168 ] {e2959088-d76d-4b5b-913d-555b91da5e18} executeQuery: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, find number failed, str: "yr2mo".: In scope SELECT parseTimeDelta('yr2mo'). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50566) (comment: 02354_parse_timedelta.sql) (in query: SELECT parseTimeDelta('yr2mo');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c1700f 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:40.679472 [ 7168 ] {e2959088-d76d-4b5b-913d-555b91da5e18} TCPHandler: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, find number failed, str: "yr2mo".: In scope SELECT parseTimeDelta('yr2mo'). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c1700f 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:40.981264 [ 7168 ] {6af6f02c-9c6a-4335-95a5-81e5d92c88aa} executeQuery: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, find number after '.' failed, str: "1.yr2mo".: In scope SELECT parseTimeDelta('1.yr2mo'). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50566) (comment: 02354_parse_timedelta.sql) (in query: SELECT parseTimeDelta('1.yr2mo');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c17139 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:40.986067 [ 7168 ] {6af6f02c-9c6a-4335-95a5-81e5d92c88aa} TCPHandler: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, find number after '.' failed, str: "1.yr2mo".: In scope SELECT parseTimeDelta('1.yr2mo'). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c17139 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:41.126749 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:41.168321 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:41.176107 [ 7168 ] {82fd277a-d77d-4722-8137-6d6ac1e0a49b} executeQuery: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, find unit failed, str: "1-yr".: In scope SELECT parseTimeDelta('1-yr'). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50566) (comment: 02354_parse_timedelta.sql) (in query: SELECT parseTimeDelta('1-yr');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c16edb 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:41.191211 [ 7168 ] {82fd277a-d77d-4722-8137-6d6ac1e0a49b} TCPHandler: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, find unit failed, str: "1-yr".: In scope SELECT parseTimeDelta('1-yr'). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c16edb 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:41.239470 [ 7168 ] {8f5d1186-04b6-4d37-94e7-34c979005424} executeQuery: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, find unit failed, str: "1 1yr".: In scope SELECT parseTimeDelta('1 1yr'). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50566) (comment: 02354_parse_timedelta.sql) (in query: SELECT parseTimeDelta('1 1yr');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c16edb 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:41.243320 [ 7168 ] {8f5d1186-04b6-4d37-94e7-34c979005424} TCPHandler: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, find unit failed, str: "1 1yr".: In scope SELECT parseTimeDelta('1 1yr'). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c16edb 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:41.462159 [ 7168 ] {d8bc06d8-62ad-4545-b9fe-94fa785466ed} executeQuery: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, parse unit failed: "yyr".: In scope SELECT parseTimeDelta('1yyr'). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50566) (comment: 02354_parse_timedelta.sql) (in query: SELECT parseTimeDelta('1yyr');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl::type, std::type_identity>&>::type>, String&&, std::basic_string_view>&) @ 0x0000000016c1991a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c16b72 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:41.465371 [ 7168 ] {d8bc06d8-62ad-4545-b9fe-94fa785466ed} TCPHandler: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, parse unit failed: "yyr".: In scope SELECT parseTimeDelta('1yyr'). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl::type, std::type_identity>&>::type>, String&&, std::basic_string_view>&) @ 0x0000000016c1991a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c16b72 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:41.500953 [ 7168 ] {1434cc69-fb26-4dc1-a264-c7c63589e512} executeQuery: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, find number failed, str: "1yr-2mo-4w + 12 days, 3 hours : 1 minute ;. 33 seconds".: In scope SELECT parseTimeDelta('1yr-2mo-4w + 12 days, 3 hours : 1 minute ;. 33 seconds'). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50566) (comment: 02354_parse_timedelta.sql) (in query: SELECT parseTimeDelta('1yr-2mo-4w + 12 days, 3 hours : 1 minute ;. 33 seconds');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c1700f 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:41.505605 [ 7168 ] {1434cc69-fb26-4dc1-a264-c7c63589e512} TCPHandler: Code: 36. DB::Exception: Invalid expression for function parseTimeDelta, find number failed, str: "1yr-2mo-4w + 12 days, 3 hours : 1 minute ;. 33 seconds".: In scope SELECT parseTimeDelta('1yr-2mo-4w + 12 days, 3 hours : 1 minute ;. 33 seconds'). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionParseTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000016c1700f 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:09:41.541920 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:41.556585 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:42.061739 [ 7176 ] {415e0766-e1cf-468d-b21c-bd2ab8bb5b3b} executeQuery: Code: 57. DB::Exception: Table test_3.nullable_00457 already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50602) (comment: 00457_log_tinylog_stripelog_nullable.sql) (in query: CREATE TABLE nullable_00457 (s String, ns Nullable(String), narr Array(Nullable(UInt64))) ENGINE = TinyLog;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:42.065168 [ 7176 ] {415e0766-e1cf-468d-b21c-bd2ab8bb5b3b} TCPHandler: Code: 57. DB::Exception: Table test_3.nullable_00457 already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:42.157674 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:42.189382 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:42.533004 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:42.609229 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:43.159688 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:43.243488 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:44.165159 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:44.205757 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:44.262642 [ 3361 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {847a217e-a513-42bb-858f-ba3e2c92857a::202509_1_21_4}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001adfc17a 4. ./build_docker/./src/Common/MemoryTracker.cpp:187: MemoryTracker::injectFault() const @ 0x000000001adf8d1a 5. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x000000002cc30a65 6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f47b 7. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:44.683587 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:44.716853 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:45.378241 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:45.488650 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:46.148946 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:46.161090 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:47.231268 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:47.269321 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:47.562747 [ 2707 ] {2854011e-ba2f-4424-807f-05ff1c78f27d} executeQuery: Code: 42. DB::Exception: Aggregate function groupBitAnd requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT BIT_AND([], [], []);) 2025.09.25 21:09:47.583161 [ 2707 ] {2854011e-ba2f-4424-807f-05ff1c78f27d} TCPHandler: Code: 42. DB::Exception: Aggregate function groupBitAnd requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:47.686232 [ 2707 ] {02d82453-6e1d-4ccd-ac1e-752b617d2438} executeQuery: Code: 42. DB::Exception: Aggregate function groupBitOr requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT BIT_OR([], [], []);) 2025.09.25 21:09:47.690222 [ 2707 ] {02d82453-6e1d-4ccd-ac1e-752b617d2438} TCPHandler: Code: 42. DB::Exception: Aggregate function groupBitOr requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:47.742219 [ 2707 ] {b78219b9-7f7d-4a20-bf77-404dfa461f8a} executeQuery: Code: 42. DB::Exception: Aggregate function groupBitXor requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT BIT_XOR([], [], []);) 2025.09.25 21:09:47.753676 [ 2707 ] {b78219b9-7f7d-4a20-bf77-404dfa461f8a} TCPHandler: Code: 42. DB::Exception: Aggregate function groupBitXor requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:47.808020 [ 2707 ] {263fb0f0-be30-44d3-af38-a5a5d72b6806} executeQuery: Code: 42. DB::Exception: Number of arguments for function BLAKE3 doesn't match: passed 3, should be 1: In scope SELECT BLAKE3([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT BLAKE3([], [], []);) 2025.09.25 21:09:47.816882 [ 2707 ] {263fb0f0-be30-44d3-af38-a5a5d72b6806} TCPHandler: Code: 42. DB::Exception: Number of arguments for function BLAKE3 doesn't match: passed 3, should be 1: In scope SELECT BLAKE3([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:47.887416 [ 2707 ] {b850f2da-42c0-448a-93e4-1c564eb7361e} executeQuery: Code: 42. DB::Exception: Number of arguments for function lengthUTF8 doesn't match: passed 3, should be 1: In scope SELECT CHARACTER_LENGTH([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT CHARACTER_LENGTH([], [], []);) 2025.09.25 21:09:47.893193 [ 2707 ] {b850f2da-42c0-448a-93e4-1c564eb7361e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function lengthUTF8 doesn't match: passed 3, should be 1: In scope SELECT CHARACTER_LENGTH([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:47.940005 [ 2707 ] {974f0fbb-2b96-4ee8-a347-26f899780db3} executeQuery: Code: 42. DB::Exception: Number of arguments for function lengthUTF8 doesn't match: passed 3, should be 1: In scope SELECT CHAR_LENGTH([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT CHAR_LENGTH([], [], []);) 2025.09.25 21:09:47.943754 [ 2707 ] {974f0fbb-2b96-4ee8-a347-26f899780db3} TCPHandler: Code: 42. DB::Exception: Number of arguments for function lengthUTF8 doesn't match: passed 3, should be 1: In scope SELECT CHAR_LENGTH([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:47.996122 [ 2707 ] {6a4b6643-a344-497c-bc70-1cf979a19073} executeQuery: Code: 42. DB::Exception: Aggregate function covarPop requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT COVAR_POP([], [], []);) 2025.09.25 21:09:47.999166 [ 2707 ] {6a4b6643-a344-497c-bc70-1cf979a19073} TCPHandler: Code: 42. DB::Exception: Aggregate function covarPop requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:48.075943 [ 2707 ] {5df1def1-0604-4d8c-91b6-aa7d19a9d08f} executeQuery: Code: 42. DB::Exception: Aggregate function covarSamp requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT COVAR_SAMP([], [], []);) 2025.09.25 21:09:48.080216 [ 2707 ] {5df1def1-0604-4d8c-91b6-aa7d19a9d08f} TCPHandler: Code: 42. DB::Exception: Aggregate function covarSamp requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:48.127305 [ 2707 ] {03954d2b-c4ad-4aac-9159-8323024ea284} executeQuery: Code: 42. DB::Exception: Number of arguments for function CRC32 doesn't match: passed 3, should be 1: In scope SELECT CRC32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT CRC32([], [], []);) 2025.09.25 21:09:48.130619 [ 2707 ] {03954d2b-c4ad-4aac-9159-8323024ea284} TCPHandler: Code: 42. DB::Exception: Number of arguments for function CRC32 doesn't match: passed 3, should be 1: In scope SELECT CRC32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:48.169387 [ 2707 ] {b6c77ce5-a459-4494-8102-e538b1eadd14} executeQuery: Code: 42. DB::Exception: Number of arguments for function CRC32IEEE doesn't match: passed 3, should be 1: In scope SELECT CRC32IEEE([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT CRC32IEEE([], [], []);) 2025.09.25 21:09:48.172251 [ 2707 ] {b6c77ce5-a459-4494-8102-e538b1eadd14} TCPHandler: Code: 42. DB::Exception: Number of arguments for function CRC32IEEE doesn't match: passed 3, should be 1: In scope SELECT CRC32IEEE([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:48.227398 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:48.230302 [ 2707 ] {b1dc149e-4819-43a2-b795-dd4385359bf0} executeQuery: Code: 42. DB::Exception: Number of arguments for function CRC64 doesn't match: passed 3, should be 1: In scope SELECT CRC64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT CRC64([], [], []);) 2025.09.25 21:09:48.234738 [ 2707 ] {b1dc149e-4819-43a2-b795-dd4385359bf0} TCPHandler: Code: 42. DB::Exception: Number of arguments for function CRC64 doesn't match: passed 3, should be 1: In scope SELECT CRC64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:48.243953 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:48.284667 [ 2707 ] {38e88e59-3fb3-45aa-bd83-8d33d6e528b3} executeQuery: Code: 42. DB::Exception: Number of arguments for function currentDatabase doesn't match: passed 3, should be 0: In scope SELECT DATABASE([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT DATABASE([], [], []);) 2025.09.25 21:09:48.295363 [ 2707 ] {38e88e59-3fb3-45aa-bd83-8d33d6e528b3} TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentDatabase doesn't match: passed 3, should be 0: In scope SELECT DATABASE([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:48.336925 [ 2707 ] {8839b8ac-2b7b-4083-9bde-6903ccb4dccf} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDate'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT DATE([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT DATE([], [], []);) 2025.09.25 21:09:48.339870 [ 2707 ] {8839b8ac-2b7b-4083-9bde-6903ccb4dccf} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDate'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT DATE([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:48.370882 [ 2707 ] {38f7c5a7-7185-4c01-b48a-cc31d23efb8b} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'unit' to function 'dateDiff'. Expected: String, got: Array(Nothing): In scope SELECT dateDiff([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT DATE_DIFF([], [], []);) 2025.09.25 21:09:48.373372 [ 2707 ] {38f7c5a7-7185-4c01-b48a-cc31d23efb8b} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'unit' to function 'dateDiff'. Expected: String, got: Array(Nothing): In scope SELECT dateDiff([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:48.420171 [ 2707 ] {dd843d12-2017-47fc-96d2-6994a178e020} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function formatDateTime. Should be a date or a date with time: In scope SELECT DATE_FORMAT([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT DATE_FORMAT([], [], []);) 2025.09.25 21:09:48.422841 [ 2707 ] {dd843d12-2017-47fc-96d2-6994a178e020} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function formatDateTime. Should be a date or a date with time: In scope SELECT DATE_FORMAT([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:48.561302 [ 2707 ] {a608b0ce-1563-442f-8892-90af9355de72} executeQuery: Code: 43. DB::Exception: First argument for function dateTrunc must be constant string: name of datepart: In scope SELECT DATE_TRUNC([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT DATE_TRUNC([], [], []);) 2025.09.25 21:09:48.568633 [ 2707 ] {a608b0ce-1563-442f-8892-90af9355de72} TCPHandler: Code: 43. DB::Exception: First argument for function dateTrunc must be constant string: name of datepart: In scope SELECT DATE_TRUNC([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:48.654782 [ 2707 ] {a5a074d1-091a-41d9-aac5-ab6e4af49f4d} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDayOfMonth doesn't match: passed 3, should be 1 or 2: In scope SELECT DAY([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT DAY([], [], []);) 2025.09.25 21:09:48.668598 [ 2707 ] {a5a074d1-091a-41d9-aac5-ab6e4af49f4d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDayOfMonth doesn't match: passed 3, should be 1 or 2: In scope SELECT DAY([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:48.807600 [ 2707 ] {55b7faf8-0181-4538-90d5-c6a984e03790} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDayOfMonth doesn't match: passed 3, should be 1 or 2: In scope SELECT DAYOFMONTH([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT DAYOFMONTH([], [], []);) 2025.09.25 21:09:48.812188 [ 2707 ] {55b7faf8-0181-4538-90d5-c6a984e03790} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDayOfMonth doesn't match: passed 3, should be 1 or 2: In scope SELECT DAYOFMONTH([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:48.903642 [ 2707 ] {40e8e3be-e169-4096-aaf3-82c375eb2e9c} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function toDayOfWeek. Must be Date, Date32, DateTime or DateTime64: In scope SELECT DAYOFWEEK([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT DAYOFWEEK([], [], []);) 2025.09.25 21:09:48.919408 [ 2707 ] {40e8e3be-e169-4096-aaf3-82c375eb2e9c} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function toDayOfWeek. Must be Date, Date32, DateTime or DateTime64: In scope SELECT DAYOFWEEK([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:49.069853 [ 2707 ] {695b6f57-ccf3-4bf1-b0c2-03e048cd9b37} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDayOfYear doesn't match: passed 3, should be 1 or 2: In scope SELECT DAYOFYEAR([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT DAYOFYEAR([], [], []);) 2025.09.25 21:09:49.076823 [ 2707 ] {695b6f57-ccf3-4bf1-b0c2-03e048cd9b37} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDayOfYear doesn't match: passed 3, should be 1 or 2: In scope SELECT DAYOFYEAR([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:49.186708 [ 2707 ] {fa79115f-10dc-41dc-9d89-53ccc07b0cc2} executeQuery: Code: 42. DB::Exception: Number of arguments for function formatReadableSize doesn't match: passed 3, should be 1: In scope SELECT FORMAT_BYTES([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT FORMAT_BYTES([], [], []);) 2025.09.25 21:09:49.198360 [ 2707 ] {fa79115f-10dc-41dc-9d89-53ccc07b0cc2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function formatReadableSize doesn't match: passed 3, should be 1: In scope SELECT FORMAT_BYTES([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:49.287948 [ 2707 ] {6c5cefa4-d4fc-4d64-9394-db35a9e46751} executeQuery: Code: 42. DB::Exception: Number of arguments for function FQDN doesn't match: passed 3, should be 0: In scope SELECT FQDN([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT FQDN([], [], []);) 2025.09.25 21:09:49.293989 [ 2707 ] {6c5cefa4-d4fc-4d64-9394-db35a9e46751} TCPHandler: Code: 42. DB::Exception: Number of arguments for function FQDN doesn't match: passed 3, should be 0: In scope SELECT FQDN([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:49.353445 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:49.396889 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:49.402816 [ 2707 ] {cd268e12-40f9-4245-8322-7cba427ff962} executeQuery: Code: 42. DB::Exception: Number of arguments for function base64Decode doesn't match: passed 3, should be 1: In scope SELECT FROM_BASE64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT FROM_BASE64([], [], []);) 2025.09.25 21:09:49.407141 [ 2707 ] {cd268e12-40f9-4245-8322-7cba427ff962} TCPHandler: Code: 42. DB::Exception: Number of arguments for function base64Decode doesn't match: passed 3, should be 1: In scope SELECT FROM_BASE64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:49.459746 [ 2707 ] {31a7b323-8cf5-47e5-a0cd-6694c9811040} executeQuery: Code: 42. DB::Exception: Number of arguments for function fromDaysSinceYearZero doesn't match: passed 3, should be 1: In scope SELECT FROM_DAYS([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT FROM_DAYS([], [], []);) 2025.09.25 21:09:49.462561 [ 2707 ] {31a7b323-8cf5-47e5-a0cd-6694c9811040} TCPHandler: Code: 42. DB::Exception: Number of arguments for function fromDaysSinceYearZero doesn't match: passed 3, should be 1: In scope SELECT FROM_DAYS([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:49.523197 [ 2707 ] {ad0a73cd-b2c7-43d9-b441-263eca019541} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function fromUnixTimestamp when arguments size is 2 or 3. Should be a integer or a date with time: In scope SELECT FROM_UNIXTIME([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT FROM_UNIXTIME([], [], []);) 2025.09.25 21:09:49.535671 [ 2707 ] {ad0a73cd-b2c7-43d9-b441-263eca019541} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function fromUnixTimestamp when arguments size is 2 or 3. Should be a integer or a date with time: In scope SELECT FROM_UNIXTIME([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:49.594076 [ 2707 ] {6e2d5161-9124-49c7-9739-4c1c27309f5f} executeQuery: Code: 42. DB::Exception: Number of arguments for function toHour doesn't match: passed 3, should be 1 or 2: In scope SELECT HOUR([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT HOUR([], [], []);) 2025.09.25 21:09:49.597802 [ 2707 ] {6e2d5161-9124-49c7-9739-4c1c27309f5f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toHour doesn't match: passed 3, should be 1 or 2: In scope SELECT HOUR([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:49.658392 [ 2707 ] {545a4e1d-d6f6-4ffa-be64-f1ac2ebcaade} executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv6StringToNum doesn't match: passed 3, should be 1: In scope SELECT INET6_ATON([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT INET6_ATON([], [], []);) 2025.09.25 21:09:49.661695 [ 2707 ] {545a4e1d-d6f6-4ffa-be64-f1ac2ebcaade} TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv6StringToNum doesn't match: passed 3, should be 1: In scope SELECT INET6_ATON([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:49.677257 [ 7168 ] {99a431ad-fceb-460c-818e-9c248076313c} DatabaseReplicated (test_5): It's not recommended to explicitly specify zookeeper_path and replica_name in ReplicatedMergeTree arguments 2025.09.25 21:09:49.717550 [ 2707 ] {51b1bfe9-7293-4ad9-87d0-a7f7ed9947dd} executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv6NumToString doesn't match: passed 3, should be 1: In scope SELECT INET6_NTOA([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT INET6_NTOA([], [], []);) 2025.09.25 21:09:49.720377 [ 2707 ] {51b1bfe9-7293-4ad9-87d0-a7f7ed9947dd} TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv6NumToString doesn't match: passed 3, should be 1: In scope SELECT INET6_NTOA([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:49.763651 [ 2707 ] {d3e2d605-a990-4990-8cd1-ed335839f0db} executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4StringToNum doesn't match: passed 3, should be 1: In scope SELECT INET_ATON([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT INET_ATON([], [], []);) 2025.09.25 21:09:49.784255 [ 2707 ] {d3e2d605-a990-4990-8cd1-ed335839f0db} TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4StringToNum doesn't match: passed 3, should be 1: In scope SELECT INET_ATON([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:49.798182 [ 7168 ] {99a431ad-fceb-460c-818e-9c248076313c} executeQuery: Code: 80. DB::Exception: Explicit zookeeper_path and replica_name are specified in ReplicatedMergeTree arguments. If you really want to specify it explicitly, then you should use some macros to distinguish different shards and replicas. (INCORRECT_QUERY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50658) (comment: 00753_alter_attach.sql) (in query: CREATE TABLE replicated_table_detach_all1 (`id` UInt64, `Data` String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test_00753_{database}/replicated_table_detach_all', '1') PARTITION BY id ORDER BY id SETTINGS index_granularity = 61490, min_bytes_for_wide_part = 440922076, ratio_of_defaults_for_sparse_serialization = 1., replace_long_file_name_to_hash = true, max_file_name_length = 0, min_bytes_for_full_part_storage = 0, compact_parts_max_bytes_to_buffer = 449386098, compact_parts_max_granules_to_buffer = 109, compact_parts_merge_max_bytes_to_prefetch_part = 4751802, merge_max_block_size = 6871, old_parts_lifetime = 480., prefer_fetch_merged_part_size_threshold = 10737418240, vertical_merge_algorithm_min_rows_to_activate = 1000000, vertical_merge_algorithm_min_columns_to_activate = 1, min_merge_bytes_to_use_direct_io = 4669302753, index_granularity_bytes = 14167832, concurrent_part_removal_threshold = 0, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = false, marks_compress_block_size = 23471, primary_key_compress_block_size = 95243), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:878: DB::DatabaseReplicated::checkTableEngine(DB::ASTCreateQuery const&, DB::ASTStorage&, std::shared_ptr) const @ 0x0000000027d5afc7 5. ./build_docker/./src/Databases/DatabaseReplicated.cpp:957: DB::DatabaseReplicated::checkQueryValid(std::shared_ptr const&, std::shared_ptr) const @ 0x0000000027d5e336 6. ./build_docker/./src/Databases/DatabaseReplicated.cpp:999: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5f0de 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:49.802706 [ 7168 ] {99a431ad-fceb-460c-818e-9c248076313c} TCPHandler: Code: 80. DB::Exception: Explicit zookeeper_path and replica_name are specified in ReplicatedMergeTree arguments. If you really want to specify it explicitly, then you should use some macros to distinguish different shards and replicas. (INCORRECT_QUERY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:878: DB::DatabaseReplicated::checkTableEngine(DB::ASTCreateQuery const&, DB::ASTStorage&, std::shared_ptr) const @ 0x0000000027d5afc7 5. ./build_docker/./src/Databases/DatabaseReplicated.cpp:957: DB::DatabaseReplicated::checkQueryValid(std::shared_ptr const&, std::shared_ptr) const @ 0x0000000027d5e336 6. ./build_docker/./src/Databases/DatabaseReplicated.cpp:999: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5f0de 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:49.857614 [ 2707 ] {c8c23d0e-1b13-4bfb-abbb-32b270a9a515} executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4NumToString doesn't match: passed 3, should be 1: In scope SELECT INET_NTOA([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT INET_NTOA([], [], []);) 2025.09.25 21:09:49.864412 [ 2707 ] {c8c23d0e-1b13-4bfb-abbb-32b270a9a515} TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4NumToString doesn't match: passed 3, should be 1: In scope SELECT INET_NTOA([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:49.931432 [ 7178 ] {862211a6-98ba-4b54-9154-1505bee1404e} executeQuery: Code: 57. DB::Exception: Table test.hits_stripelog already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34746) (comment: 00077_log_tinylog_stripelog.sql) (in query: CREATE TABLE test.hits_stripelog (CounterID UInt32, AdvEngineID UInt8, RegionID UInt32, SearchPhrase String, UserID UInt64) ENGINE = StripeLog;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:49.941573 [ 2707 ] {4f3d03ef-913f-4810-9987-eb35834ffd67} executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4CIDRToRange doesn't match: passed 3, should be 2: In scope SELECT IPv4CIDRToRange([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT IPv4CIDRToRange([], [], []);) 2025.09.25 21:09:49.942516 [ 7178 ] {862211a6-98ba-4b54-9154-1505bee1404e} TCPHandler: Code: 57. DB::Exception: Table test.hits_stripelog already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:49.960247 [ 2707 ] {4f3d03ef-913f-4810-9987-eb35834ffd67} TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4CIDRToRange doesn't match: passed 3, should be 2: In scope SELECT IPv4CIDRToRange([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:50.027697 [ 2707 ] {4c645a36-21ed-4199-a058-64b7df7d7bdc} executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4NumToString doesn't match: passed 3, should be 1: In scope SELECT IPv4NumToString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT IPv4NumToString([], [], []);) 2025.09.25 21:09:50.035619 [ 2707 ] {4c645a36-21ed-4199-a058-64b7df7d7bdc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4NumToString doesn't match: passed 3, should be 1: In scope SELECT IPv4NumToString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:50.128020 [ 2707 ] {64bdfd50-fc46-43df-ac01-c3cb3be5c292} executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4NumToStringClassC doesn't match: passed 3, should be 1: In scope SELECT IPv4NumToStringClassC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT IPv4NumToStringClassC([], [], []);) 2025.09.25 21:09:50.133816 [ 2707 ] {64bdfd50-fc46-43df-ac01-c3cb3be5c292} TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4NumToStringClassC doesn't match: passed 3, should be 1: In scope SELECT IPv4NumToStringClassC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:50.197813 [ 2707 ] {6e5ddd5a-df55-453a-b3fb-75ae0cebae4b} executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4StringToNum doesn't match: passed 3, should be 1: In scope SELECT IPv4StringToNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT IPv4StringToNum([], [], []);) 2025.09.25 21:09:50.203562 [ 2707 ] {6e5ddd5a-df55-453a-b3fb-75ae0cebae4b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4StringToNum doesn't match: passed 3, should be 1: In scope SELECT IPv4StringToNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:50.250828 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:50.259338 [ 2707 ] {16eb0f4f-e12c-448d-8eac-32a2f04bc3e5} executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4StringToNumOrDefault doesn't match: passed 3, should be 1: In scope SELECT IPv4StringToNumOrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT IPv4StringToNumOrDefault([], [], []);) 2025.09.25 21:09:50.274740 [ 2707 ] {16eb0f4f-e12c-448d-8eac-32a2f04bc3e5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4StringToNumOrDefault doesn't match: passed 3, should be 1: In scope SELECT IPv4StringToNumOrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:50.295064 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:50.337621 [ 2707 ] {fe264e4a-29b1-4fad-8c17-a9bf86ed6f72} executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4StringToNumOrNull doesn't match: passed 3, should be 1: In scope SELECT IPv4StringToNumOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT IPv4StringToNumOrNull([], [], []);) 2025.09.25 21:09:50.341031 [ 2707 ] {fe264e4a-29b1-4fad-8c17-a9bf86ed6f72} TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4StringToNumOrNull doesn't match: passed 3, should be 1: In scope SELECT IPv4StringToNumOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:50.377363 [ 2707 ] {6653ff12-bcd9-4cde-83e0-0e18d552edef} executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv4ToIPv6 doesn't match: passed 3, should be 1: In scope SELECT IPv4ToIPv6([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT IPv4ToIPv6([], [], []);) 2025.09.25 21:09:50.381702 [ 2707 ] {6653ff12-bcd9-4cde-83e0-0e18d552edef} TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv4ToIPv6 doesn't match: passed 3, should be 1: In scope SELECT IPv4ToIPv6([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:50.526273 [ 2707 ] {39a32c00-5fa9-4a2a-a171-a11e45171c1d} executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv6CIDRToRange doesn't match: passed 3, should be 2: In scope SELECT IPv6CIDRToRange([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT IPv6CIDRToRange([], [], []);) 2025.09.25 21:09:50.532623 [ 2707 ] {39a32c00-5fa9-4a2a-a171-a11e45171c1d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv6CIDRToRange doesn't match: passed 3, should be 2: In scope SELECT IPv6CIDRToRange([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:50.650634 [ 7178 ] {c536c898-1559-4e9c-85a3-dd9c03ec2dc6} executeQuery: Code: 43. DB::Exception: Illegal types Decimal(76, 3) and Date of arguments of function intDiv: In scope SELECT intDiv(CAST('1.0', 'Decimal256(3)'), today()). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34752) (comment: 03002_int_div_decimal_with_date_bug.sql) (in query: SELECT intDiv(CAST('1.0', 'Decimal256(3)'), today());), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionBinaryArithmetic::getReturnTypeImplStatic(std::vector, std::allocator>> const&, std::shared_ptr) @ 0x0000000012854591 5. DB::BinaryArithmeticOverloadResolver::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x00000000121dd3c9 6. DB::IFunctionOverloadResolver::getReturnTypeImpl(std::vector> const&) const @ 0x000000000c2b5b32 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:09:50.653954 [ 7178 ] {c536c898-1559-4e9c-85a3-dd9c03ec2dc6} TCPHandler: Code: 43. DB::Exception: Illegal types Decimal(76, 3) and Date of arguments of function intDiv: In scope SELECT intDiv(CAST('1.0', 'Decimal256(3)'), today()). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionBinaryArithmetic::getReturnTypeImplStatic(std::vector, std::allocator>> const&, std::shared_ptr) @ 0x0000000012854591 5. DB::BinaryArithmeticOverloadResolver::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x00000000121dd3c9 6. DB::IFunctionOverloadResolver::getReturnTypeImpl(std::vector> const&) const @ 0x000000000c2b5b32 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:09:50.676891 [ 2707 ] {cc6c4276-8850-45a7-9bfd-43ecad84618f} executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv6NumToString doesn't match: passed 3, should be 1: In scope SELECT IPv6NumToString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT IPv6NumToString([], [], []);) 2025.09.25 21:09:50.683828 [ 2707 ] {cc6c4276-8850-45a7-9bfd-43ecad84618f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv6NumToString doesn't match: passed 3, should be 1: In scope SELECT IPv6NumToString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:50.709680 [ 7178 ] {bea9a843-32c0-4f59-a1e6-92a9d856fab5} executeQuery: Code: 43. DB::Exception: Illegal types Decimal(76, 3) and Date of arguments of function intDiv: In scope SELECT intDiv(CAST('1.0', 'Decimal256(3)'), toDate('2023-01-02')). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34752) (comment: 03002_int_div_decimal_with_date_bug.sql) (in query: SELECT intDiv(CAST('1.0', 'Decimal256(3)'), toDate('2023-01-02'));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionBinaryArithmetic::getReturnTypeImplStatic(std::vector, std::allocator>> const&, std::shared_ptr) @ 0x0000000012854591 5. DB::BinaryArithmeticOverloadResolver::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x00000000121dd3c9 6. DB::IFunctionOverloadResolver::getReturnTypeImpl(std::vector> const&) const @ 0x000000000c2b5b32 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:09:50.715996 [ 7178 ] {bea9a843-32c0-4f59-a1e6-92a9d856fab5} TCPHandler: Code: 43. DB::Exception: Illegal types Decimal(76, 3) and Date of arguments of function intDiv: In scope SELECT intDiv(CAST('1.0', 'Decimal256(3)'), toDate('2023-01-02')). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionBinaryArithmetic::getReturnTypeImplStatic(std::vector, std::allocator>> const&, std::shared_ptr) @ 0x0000000012854591 5. DB::BinaryArithmeticOverloadResolver::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x00000000121dd3c9 6. DB::IFunctionOverloadResolver::getReturnTypeImpl(std::vector> const&) const @ 0x000000000c2b5b32 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:09:50.747989 [ 2707 ] {75915adf-99d5-417e-babd-dd64e2270b01} executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv6StringToNum doesn't match: passed 3, should be 1: In scope SELECT IPv6StringToNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT IPv6StringToNum([], [], []);) 2025.09.25 21:09:50.762345 [ 2707 ] {75915adf-99d5-417e-babd-dd64e2270b01} TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv6StringToNum doesn't match: passed 3, should be 1: In scope SELECT IPv6StringToNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:50.807912 [ 7178 ] {7bc9cfce-bb35-45f5-8047-2a06f959cfc1} executeQuery: Code: 43. DB::Exception: Illegal types Decimal(76, 2) and Date32 of arguments of function intDiv: In scope SELECT intDiv(CAST('1.0', 'Decimal256(2)'), toDate32('2023-01-02 12:12:12')). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34752) (comment: 03002_int_div_decimal_with_date_bug.sql) (in query: SELECT intDiv(CAST('1.0', 'Decimal256(2)'), toDate32('2023-01-02 12:12:12'));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionBinaryArithmetic::getReturnTypeImplStatic(std::vector, std::allocator>> const&, std::shared_ptr) @ 0x0000000012854591 5. DB::BinaryArithmeticOverloadResolver::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x00000000121dd3c9 6. DB::IFunctionOverloadResolver::getReturnTypeImpl(std::vector> const&) const @ 0x000000000c2b5b32 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:09:50.814284 [ 7178 ] {7bc9cfce-bb35-45f5-8047-2a06f959cfc1} TCPHandler: Code: 43. DB::Exception: Illegal types Decimal(76, 2) and Date32 of arguments of function intDiv: In scope SELECT intDiv(CAST('1.0', 'Decimal256(2)'), toDate32('2023-01-02 12:12:12')). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionBinaryArithmetic::getReturnTypeImplStatic(std::vector, std::allocator>> const&, std::shared_ptr) @ 0x0000000012854591 5. DB::BinaryArithmeticOverloadResolver::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x00000000121dd3c9 6. DB::IFunctionOverloadResolver::getReturnTypeImpl(std::vector> const&) const @ 0x000000000c2b5b32 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:09:50.826097 [ 2707 ] {f9c6bbbd-2f13-436e-a278-e6a9b50a05ab} executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv6StringToNumOrDefault doesn't match: passed 3, should be 1: In scope SELECT IPv6StringToNumOrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT IPv6StringToNumOrDefault([], [], []);) 2025.09.25 21:09:50.833881 [ 2707 ] {f9c6bbbd-2f13-436e-a278-e6a9b50a05ab} TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv6StringToNumOrDefault doesn't match: passed 3, should be 1: In scope SELECT IPv6StringToNumOrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:50.957292 [ 7178 ] {d20a0eac-dcf8-4657-a1d9-9ad923e58d6a} executeQuery: Code: 43. DB::Exception: Illegal types Decimal(76, 2) and DateTime of arguments of function intDiv: In scope SELECT intDiv(CAST('1.0', 'Decimal256(2)'), toDateTime('2023-01-02 12:12:12')). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34752) (comment: 03002_int_div_decimal_with_date_bug.sql) (in query: SELECT intDiv(CAST('1.0', 'Decimal256(2)'), toDateTime('2023-01-02 12:12:12'));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionBinaryArithmetic::getReturnTypeImplStatic(std::vector, std::allocator>> const&, std::shared_ptr) @ 0x0000000012854591 5. DB::BinaryArithmeticOverloadResolver::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x00000000121dd3c9 6. DB::IFunctionOverloadResolver::getReturnTypeImpl(std::vector> const&) const @ 0x000000000c2b5b32 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:09:50.961471 [ 7178 ] {d20a0eac-dcf8-4657-a1d9-9ad923e58d6a} TCPHandler: Code: 43. DB::Exception: Illegal types Decimal(76, 2) and DateTime of arguments of function intDiv: In scope SELECT intDiv(CAST('1.0', 'Decimal256(2)'), toDateTime('2023-01-02 12:12:12')). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionBinaryArithmetic::getReturnTypeImplStatic(std::vector, std::allocator>> const&, std::shared_ptr) @ 0x0000000012854591 5. DB::BinaryArithmeticOverloadResolver::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x00000000121dd3c9 6. DB::IFunctionOverloadResolver::getReturnTypeImpl(std::vector> const&) const @ 0x000000000c2b5b32 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:09:50.981177 [ 2707 ] {aa394793-136d-4a7a-b685-cb7356091ce2} executeQuery: Code: 42. DB::Exception: Number of arguments for function IPv6StringToNumOrNull doesn't match: passed 3, should be 1: In scope SELECT IPv6StringToNumOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT IPv6StringToNumOrNull([], [], []);) 2025.09.25 21:09:50.987878 [ 2707 ] {aa394793-136d-4a7a-b685-cb7356091ce2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function IPv6StringToNumOrNull doesn't match: passed 3, should be 1: In scope SELECT IPv6StringToNumOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:51.030414 [ 7178 ] {dd938999-ba21-434b-a577-b58b73160faf} executeQuery: Code: 43. DB::Exception: Illegal types Decimal(76, 2) and DateTime64(3) of arguments of function intDiv: In scope SELECT intDiv(CAST('1.0', 'Decimal256(2)'), toDateTime64('2023-01-02 12:12:12.002', 3)). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34752) (comment: 03002_int_div_decimal_with_date_bug.sql) (in query: SELECT intDiv(CAST('1.0', 'Decimal256(2)'), toDateTime64('2023-01-02 12:12:12.002', 3));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionBinaryArithmetic::getReturnTypeImplStatic(std::vector, std::allocator>> const&, std::shared_ptr) @ 0x0000000012854591 5. DB::BinaryArithmeticOverloadResolver::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x00000000121dd3c9 6. DB::IFunctionOverloadResolver::getReturnTypeImpl(std::vector> const&) const @ 0x000000000c2b5b32 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:09:51.047046 [ 7178 ] {dd938999-ba21-434b-a577-b58b73160faf} TCPHandler: Code: 43. DB::Exception: Illegal types Decimal(76, 2) and DateTime64(3) of arguments of function intDiv: In scope SELECT intDiv(CAST('1.0', 'Decimal256(2)'), toDateTime64('2023-01-02 12:12:12.002', 3)). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionBinaryArithmetic::getReturnTypeImplStatic(std::vector, std::allocator>> const&, std::shared_ptr) @ 0x0000000012854591 5. DB::BinaryArithmeticOverloadResolver::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x00000000121dd3c9 6. DB::IFunctionOverloadResolver::getReturnTypeImpl(std::vector> const&) const @ 0x000000000c2b5b32 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:09:51.085020 [ 2707 ] {072d2141-cd66-4701-826c-e3018865185f} executeQuery: Code: 42. DB::Exception: Number of arguments for function JSONAllPaths doesn't match: passed 3, should be 1: In scope SELECT JSONAllPaths([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONAllPaths([], [], []);) 2025.09.25 21:09:51.101548 [ 2707 ] {072d2141-cd66-4701-826c-e3018865185f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function JSONAllPaths doesn't match: passed 3, should be 1: In scope SELECT JSONAllPaths([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:51.184115 [ 2707 ] {236475fd-b50f-4adc-b400-12de97b019fb} executeQuery: Code: 42. DB::Exception: Number of arguments for function JSONAllPathsWithTypes doesn't match: passed 3, should be 1: In scope SELECT JSONAllPathsWithTypes([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONAllPathsWithTypes([], [], []);) 2025.09.25 21:09:51.194235 [ 2707 ] {236475fd-b50f-4adc-b400-12de97b019fb} TCPHandler: Code: 42. DB::Exception: Number of arguments for function JSONAllPathsWithTypes doesn't match: passed 3, should be 1: In scope SELECT JSONAllPathsWithTypes([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:51.259038 [ 2707 ] {e868be67-6eab-4008-9da0-a6d4e97a2f8e} executeQuery: Code: 42. DB::Exception: Number of arguments for function JSONArrayLength doesn't match: passed 3, should be 1: In scope SELECT JSONArrayLength([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONArrayLength([], [], []);) 2025.09.25 21:09:51.270506 [ 2707 ] {e868be67-6eab-4008-9da0-a6d4e97a2f8e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function JSONArrayLength doesn't match: passed 3, should be 1: In scope SELECT JSONArrayLength([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:51.460523 [ 2707 ] {c65236e8-c4ec-4e2e-835b-7d5880d4ff6a} executeQuery: Code: 42. DB::Exception: Number of arguments for function JSONDynamicPaths doesn't match: passed 3, should be 1: In scope SELECT JSONDynamicPaths([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONDynamicPaths([], [], []);) 2025.09.25 21:09:51.467062 [ 2707 ] {c65236e8-c4ec-4e2e-835b-7d5880d4ff6a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function JSONDynamicPaths doesn't match: passed 3, should be 1: In scope SELECT JSONDynamicPaths([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:51.534513 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:51.596587 [ 2707 ] {0ce51af5-6ec1-43f0-b540-daf4cf1fc24c} executeQuery: Code: 42. DB::Exception: Number of arguments for function JSONDynamicPathsWithTypes doesn't match: passed 3, should be 1: In scope SELECT JSONDynamicPathsWithTypes([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONDynamicPathsWithTypes([], [], []);) 2025.09.25 21:09:51.608062 [ 2707 ] {0ce51af5-6ec1-43f0-b540-daf4cf1fc24c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function JSONDynamicPathsWithTypes doesn't match: passed 3, should be 1: In scope SELECT JSONDynamicPathsWithTypes([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:51.640032 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:51.713766 [ 2707 ] {b517fd2c-42ac-4f5c-9a51-6621beae0dfe} executeQuery: Code: 44. DB::Exception: The last argument of function JSONExtract should be a constant string specifying the return data type, illegal value: []: In scope SELECT JSONExtract([], [], []). (ILLEGAL_COLUMN) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONExtract([], [], []);) 2025.09.25 21:09:51.722260 [ 2707 ] {b517fd2c-42ac-4f5c-9a51-6621beae0dfe} TCPHandler: Code: 44. DB::Exception: The last argument of function JSONExtract should be a constant string specifying the return data type, illegal value: []: In scope SELECT JSONExtract([], [], []). (ILLEGAL_COLUMN) 2025.09.25 21:09:51.796141 [ 2707 ] {591ab621-67ea-43fa-a1d6-6e8d4500d164} executeQuery: Code: 43. DB::Exception: The first argument of function JSONExtractArrayRaw should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractArrayRaw([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONExtractArrayRaw([], [], []);) 2025.09.25 21:09:51.802864 [ 2707 ] {591ab621-67ea-43fa-a1d6-6e8d4500d164} TCPHandler: Code: 43. DB::Exception: The first argument of function JSONExtractArrayRaw should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractArrayRaw([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:51.868948 [ 2707 ] {269ca101-322c-4f08-95cd-bbdcccfadea4} executeQuery: Code: 43. DB::Exception: The first argument of function JSONExtractBool should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractBool([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONExtractBool([], [], []);) 2025.09.25 21:09:51.873847 [ 2707 ] {269ca101-322c-4f08-95cd-bbdcccfadea4} TCPHandler: Code: 43. DB::Exception: The first argument of function JSONExtractBool should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractBool([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:51.968260 [ 2707 ] {e8ceb3ec-1be0-41bd-a5cf-fdd079037157} executeQuery: Code: 43. DB::Exception: The first argument of function JSONExtractFloat should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractFloat([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONExtractFloat([], [], []);) 2025.09.25 21:09:51.974063 [ 2707 ] {e8ceb3ec-1be0-41bd-a5cf-fdd079037157} TCPHandler: Code: 43. DB::Exception: The first argument of function JSONExtractFloat should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractFloat([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:52.087772 [ 2707 ] {6bc8ed99-b1ab-4705-8d0c-cc39a53ae0b6} executeQuery: Code: 43. DB::Exception: The first argument of function JSONExtractInt should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractInt([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONExtractInt([], [], []);) 2025.09.25 21:09:52.098766 [ 2707 ] {6bc8ed99-b1ab-4705-8d0c-cc39a53ae0b6} TCPHandler: Code: 43. DB::Exception: The first argument of function JSONExtractInt should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractInt([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:52.200733 [ 2707 ] {b18f28a0-4279-46ff-a1d6-2130e33eeb8d} executeQuery: Code: 43. DB::Exception: The first argument of function JSONExtractKeys should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractKeys([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONExtractKeys([], [], []);) 2025.09.25 21:09:52.207226 [ 2707 ] {b18f28a0-4279-46ff-a1d6-2130e33eeb8d} TCPHandler: Code: 43. DB::Exception: The first argument of function JSONExtractKeys should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractKeys([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:52.262904 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:52.287099 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:52.297780 [ 2707 ] {ac29c50e-d0a2-49ff-9105-3c14f58ab96d} executeQuery: Code: 44. DB::Exception: The last argument of function JSONExtractKeysAndValues should be a constant string specifying the values' data type, illegal value: []: In scope SELECT JSONExtractKeysAndValues([], [], []). (ILLEGAL_COLUMN) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONExtractKeysAndValues([], [], []);) 2025.09.25 21:09:52.303186 [ 2707 ] {ac29c50e-d0a2-49ff-9105-3c14f58ab96d} TCPHandler: Code: 44. DB::Exception: The last argument of function JSONExtractKeysAndValues should be a constant string specifying the values' data type, illegal value: []: In scope SELECT JSONExtractKeysAndValues([], [], []). (ILLEGAL_COLUMN) 2025.09.25 21:09:52.369111 [ 2707 ] {6dfcaec8-5800-4d29-a406-6fb26f856a39} executeQuery: Code: 43. DB::Exception: The first argument of function JSONExtractKeysAndValuesRaw should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractKeysAndValuesRaw([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONExtractKeysAndValuesRaw([], [], []);) 2025.09.25 21:09:52.372277 [ 2707 ] {6dfcaec8-5800-4d29-a406-6fb26f856a39} TCPHandler: Code: 43. DB::Exception: The first argument of function JSONExtractKeysAndValuesRaw should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractKeysAndValuesRaw([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:52.437387 [ 2707 ] {e70db824-cb7c-4304-9340-7895546c78a8} executeQuery: Code: 43. DB::Exception: The first argument of function JSONExtractRaw should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractRaw([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONExtractRaw([], [], []);) 2025.09.25 21:09:52.469915 [ 2707 ] {e70db824-cb7c-4304-9340-7895546c78a8} TCPHandler: Code: 43. DB::Exception: The first argument of function JSONExtractRaw should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractRaw([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:52.539293 [ 7178 ] {45464dc0-644f-41a6-8773-d7673a19ba2a} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=0): (while reading column UserID): (while reading from part /var/lib/clickhouse/store/fab/fab61230-e14a-49b2-a373-9edad6fb4784/201403_3_3_0/ in table test.visits (fab61230-e14a-49b2-a373-9edad6fb4784) located on disk default of type local, from mark 0 with max_rows_to_read = 20160): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34762) (comment: 00053_replicate_segfault.sql) (in query: SELECT count() > 0 FROM (SELECT ParsedParams.Key1 AS p FROM test.visits WHERE arrayAll(y -> arrayExists(x -> y != x, p), p)) ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x000000000b581216 13. ./build_docker/./src/Compression/CompressedReadBufferBase.cpp:0: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x000000002775f390 14. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:122: DB::CompressedReadBufferFromFile::readBig(char*, unsigned long) @ 0x0000000027764cac 15. ./build_docker/./src/DataTypes/Serializations/SerializationNumber.cpp:222: DB::SerializationNumber::deserializeBinaryBulk(DB::IColumn&, DB::ReadBuffer&, unsigned long, double) const @ 0x0000000027b97730 16. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027aa00db 17. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:196: DB::MergeTreeReaderWide::readData(DB::NameAndTypePair const&, std::shared_ptr const&, COW::immutable_ptr&, unsigned long, bool, unsigned long, unsigned long, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>&, bool) @ 0x000000002c80211e 18. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:158: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c800e8d 19. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 20. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000002c7e0473 21. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbec3 22. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 23. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:53: DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002dea5374 24. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 25. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 26. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 27. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 28. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 29. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 30. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 31. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 2025.09.25 21:09:52.559239 [ 2707 ] {fa0bde26-dc01-4346-84f8-c6a72d2ae74b} executeQuery: Code: 43. DB::Exception: The first argument of function JSONExtractString should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractString([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONExtractString([], [], []);) 2025.09.25 21:09:52.563959 [ 2707 ] {fa0bde26-dc01-4346-84f8-c6a72d2ae74b} TCPHandler: Code: 43. DB::Exception: The first argument of function JSONExtractString should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractString([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:52.636843 [ 7178 ] {45464dc0-644f-41a6-8773-d7673a19ba2a} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=0): (while reading column UserID): (while reading from part /var/lib/clickhouse/store/fab/fab61230-e14a-49b2-a373-9edad6fb4784/201403_3_3_0/ in table test.visits (fab61230-e14a-49b2-a373-9edad6fb4784) located on disk default of type local, from mark 0 with max_rows_to_read = 20160): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x000000000b581216 13. ./build_docker/./src/Compression/CompressedReadBufferBase.cpp:0: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x000000002775f390 14. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:122: DB::CompressedReadBufferFromFile::readBig(char*, unsigned long) @ 0x0000000027764cac 15. ./build_docker/./src/DataTypes/Serializations/SerializationNumber.cpp:222: DB::SerializationNumber::deserializeBinaryBulk(DB::IColumn&, DB::ReadBuffer&, unsigned long, double) const @ 0x0000000027b97730 16. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027aa00db 17. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:196: DB::MergeTreeReaderWide::readData(DB::NameAndTypePair const&, std::shared_ptr const&, COW::immutable_ptr&, unsigned long, bool, unsigned long, unsigned long, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>&, bool) @ 0x000000002c80211e 18. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:158: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c800e8d 19. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 20. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000002c7e0473 21. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbec3 22. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 23. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:53: DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002dea5374 24. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 25. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 26. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 27. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 28. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 29. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 30. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 31. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 2025.09.25 21:09:52.701937 [ 2707 ] {952fee6c-58b0-421f-b96c-eb45dd47e59f} executeQuery: Code: 43. DB::Exception: The first argument of function JSONExtractUInt should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractUInt([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONExtractUInt([], [], []);) 2025.09.25 21:09:52.705789 [ 2707 ] {952fee6c-58b0-421f-b96c-eb45dd47e59f} TCPHandler: Code: 43. DB::Exception: The first argument of function JSONExtractUInt should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONExtractUInt([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:52.849731 [ 2707 ] {78050159-406e-40e8-ab5b-5694bb618655} executeQuery: Code: 43. DB::Exception: The first argument of function JSONHas should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONHas([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONHas([], [], []);) 2025.09.25 21:09:52.866379 [ 2707 ] {78050159-406e-40e8-ab5b-5694bb618655} TCPHandler: Code: 43. DB::Exception: The first argument of function JSONHas should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONHas([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:53.001738 [ 2707 ] {6d08983c-bc2d-4ecb-8514-d841409a9c54} executeQuery: Code: 43. DB::Exception: The first argument of function JSONKey should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONKey([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONKey([], [], []);) 2025.09.25 21:09:53.011661 [ 2707 ] {6d08983c-bc2d-4ecb-8514-d841409a9c54} TCPHandler: Code: 43. DB::Exception: The first argument of function JSONKey should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONKey([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:53.111773 [ 2707 ] {80ed73fc-efc2-41b8-a996-946f2a8546be} executeQuery: Code: 43. DB::Exception: The first argument of function JSONLength should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONLength([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONLength([], [], []);) 2025.09.25 21:09:53.119830 [ 2707 ] {80ed73fc-efc2-41b8-a996-946f2a8546be} TCPHandler: Code: 43. DB::Exception: The first argument of function JSONLength should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONLength([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:53.119974 [ 3619 ] {84f54395-d69a-493a-82e7-b3c3a565b336} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34778) (comment: 02155_nested_lc_defalut_bug.sql) (in query: SELECT * FROM nested_test;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:53.128855 [ 3619 ] {84f54395-d69a-493a-82e7-b3c3a565b336} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:53.176848 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:53.210455 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:53.211903 [ 2707 ] {3dcf4794-666a-44f4-a7bf-b0237bfbcebf} executeQuery: Code: 43. DB::Exception: Function JSONMergePatch requires string arguments: In scope SELECT JSONMergePatch([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONMergePatch([], [], []);) 2025.09.25 21:09:53.221033 [ 2707 ] {3dcf4794-666a-44f4-a7bf-b0237bfbcebf} TCPHandler: Code: 43. DB::Exception: Function JSONMergePatch requires string arguments: In scope SELECT JSONMergePatch([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:53.334229 [ 2707 ] {e9acc4f2-4957-4bd0-88de-09e50684bc36} executeQuery: Code: 42. DB::Exception: Number of arguments for function JSONSharedDataPaths doesn't match: passed 3, should be 1: In scope SELECT JSONSharedDataPaths([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONSharedDataPaths([], [], []);) 2025.09.25 21:09:53.347640 [ 2707 ] {e9acc4f2-4957-4bd0-88de-09e50684bc36} TCPHandler: Code: 42. DB::Exception: Number of arguments for function JSONSharedDataPaths doesn't match: passed 3, should be 1: In scope SELECT JSONSharedDataPaths([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:53.443671 [ 2707 ] {4520c579-69ac-4787-9769-7abfb472ac0f} executeQuery: Code: 42. DB::Exception: Number of arguments for function JSONSharedDataPathsWithTypes doesn't match: passed 3, should be 1: In scope SELECT JSONSharedDataPathsWithTypes([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONSharedDataPathsWithTypes([], [], []);) 2025.09.25 21:09:53.449800 [ 2707 ] {4520c579-69ac-4787-9769-7abfb472ac0f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function JSONSharedDataPathsWithTypes doesn't match: passed 3, should be 1: In scope SELECT JSONSharedDataPathsWithTypes([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:53.537216 [ 2707 ] {782e8590-2b61-48f6-be2d-6cab6eb311a8} executeQuery: Code: 43. DB::Exception: The first argument of function JSONType should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONType([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSONType([], [], []);) 2025.09.25 21:09:53.538004 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:53.542275 [ 2707 ] {782e8590-2b61-48f6-be2d-6cab6eb311a8} TCPHandler: Code: 43. DB::Exception: The first argument of function JSONType should be a string containing JSON, illegal type: Array(Nothing): In scope SELECT JSONType([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:53.550181 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:53.650604 [ 2707 ] {2b45b47d-af90-4688-bda7-b94e4cc667db} executeQuery: Code: 42. DB::Exception: Number of arguments for function JSONArrayLength doesn't match: passed 3, should be 1: In scope SELECT JSON_ARRAY_LENGTH([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSON_ARRAY_LENGTH([], [], []);) 2025.09.25 21:09:53.652950 [ 2707 ] {2b45b47d-af90-4688-bda7-b94e4cc667db} TCPHandler: Code: 42. DB::Exception: Number of arguments for function JSONArrayLength doesn't match: passed 3, should be 1: In scope SELECT JSON_ARRAY_LENGTH([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:53.700728 [ 2707 ] {7014b3f8-8fe0-4228-a229-99a60f63ed42} executeQuery: Code: 43. DB::Exception: JSONPath functions require first argument to be JSON of string, illegal type: Array(Nothing): In scope SELECT JSON_EXISTS([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSON_EXISTS([], [], []);) 2025.09.25 21:09:53.709575 [ 2707 ] {7014b3f8-8fe0-4228-a229-99a60f63ed42} TCPHandler: Code: 43. DB::Exception: JSONPath functions require first argument to be JSON of string, illegal type: Array(Nothing): In scope SELECT JSON_EXISTS([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:53.759883 [ 2707 ] {5da78123-b1c5-4271-a4c9-ccbbd6784a5f} executeQuery: Code: 43. DB::Exception: JSONPath functions require first argument to be JSON of string, illegal type: Array(Nothing): In scope SELECT JSON_QUERY([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSON_QUERY([], [], []);) 2025.09.25 21:09:53.766985 [ 2707 ] {5da78123-b1c5-4271-a4c9-ccbbd6784a5f} TCPHandler: Code: 43. DB::Exception: JSONPath functions require first argument to be JSON of string, illegal type: Array(Nothing): In scope SELECT JSON_QUERY([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:53.804708 [ 2707 ] {a492875c-585c-481f-9ac7-87594e4933ad} executeQuery: Code: 43. DB::Exception: JSONPath functions require first argument to be JSON of string, illegal type: Array(Nothing): In scope SELECT JSON_VALUE([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT JSON_VALUE([], [], []);) 2025.09.25 21:09:53.807952 [ 2707 ] {a492875c-585c-481f-9ac7-87594e4933ad} TCPHandler: Code: 43. DB::Exception: JSONPath functions require first argument to be JSON of string, illegal type: Array(Nothing): In scope SELECT JSON_VALUE([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:53.945603 [ 2707 ] {11cef6be-73cd-4d82-a7d0-27dd3a0d4517} executeQuery: Code: 42. DB::Exception: Number of arguments for function L1Distance doesn't match: passed 3, should be 2: In scope SELECT L1Distance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT L1Distance([], [], []);) 2025.09.25 21:09:53.949684 [ 2707 ] {11cef6be-73cd-4d82-a7d0-27dd3a0d4517} TCPHandler: Code: 42. DB::Exception: Number of arguments for function L1Distance doesn't match: passed 3, should be 2: In scope SELECT L1Distance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:53.984835 [ 2707 ] {a8b87133-ee45-41ef-80a5-7d77f723fa1c} executeQuery: Code: 42. DB::Exception: Number of arguments for function L1Norm doesn't match: passed 3, should be 1: In scope SELECT L1Norm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT L1Norm([], [], []);) 2025.09.25 21:09:53.988585 [ 2707 ] {a8b87133-ee45-41ef-80a5-7d77f723fa1c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function L1Norm doesn't match: passed 3, should be 1: In scope SELECT L1Norm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:54.027781 [ 2707 ] {a7c7b7cc-7125-49bc-935c-b5d63c349e10} executeQuery: Code: 42. DB::Exception: Number of arguments for function L1Normalize doesn't match: passed 3, should be 1: In scope SELECT L1Normalize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT L1Normalize([], [], []);) 2025.09.25 21:09:54.030005 [ 2707 ] {a7c7b7cc-7125-49bc-935c-b5d63c349e10} TCPHandler: Code: 42. DB::Exception: Number of arguments for function L1Normalize doesn't match: passed 3, should be 1: In scope SELECT L1Normalize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:54.064722 [ 2707 ] {eb78afb9-3666-47e8-bbc6-62a37dd69f38} executeQuery: Code: 42. DB::Exception: Number of arguments for function L2Distance doesn't match: passed 3, should be 2: In scope SELECT L2Distance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT L2Distance([], [], []);) 2025.09.25 21:09:54.067231 [ 2707 ] {eb78afb9-3666-47e8-bbc6-62a37dd69f38} TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2Distance doesn't match: passed 3, should be 2: In scope SELECT L2Distance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:54.102916 [ 2707 ] {a0f64b1a-d3e8-4c03-89df-83ce92685ffe} executeQuery: Code: 42. DB::Exception: Number of arguments for function L2Norm doesn't match: passed 3, should be 1: In scope SELECT L2Norm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT L2Norm([], [], []);) 2025.09.25 21:09:54.106203 [ 2707 ] {a0f64b1a-d3e8-4c03-89df-83ce92685ffe} TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2Norm doesn't match: passed 3, should be 1: In scope SELECT L2Norm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:54.147841 [ 2707 ] {96777009-fc57-43ef-bcab-b0ffb31f63fb} executeQuery: Code: 42. DB::Exception: Number of arguments for function L2Normalize doesn't match: passed 3, should be 1: In scope SELECT L2Normalize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT L2Normalize([], [], []);) 2025.09.25 21:09:54.150679 [ 2707 ] {96777009-fc57-43ef-bcab-b0ffb31f63fb} TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2Normalize doesn't match: passed 3, should be 1: In scope SELECT L2Normalize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:54.222463 [ 2707 ] {491897e2-e500-4d36-a261-46ebfb57e2d3} executeQuery: Code: 42. DB::Exception: Number of arguments for function L2SquaredDistance doesn't match: passed 3, should be 2: In scope SELECT L2SquaredDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT L2SquaredDistance([], [], []);) 2025.09.25 21:09:54.227047 [ 2707 ] {491897e2-e500-4d36-a261-46ebfb57e2d3} TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2SquaredDistance doesn't match: passed 3, should be 2: In scope SELECT L2SquaredDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:54.260405 [ 2707 ] {802de624-4ea2-48a8-a4a0-906029a89b2d} executeQuery: Code: 42. DB::Exception: Number of arguments for function L2SquaredNorm doesn't match: passed 3, should be 1: In scope SELECT L2SquaredNorm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT L2SquaredNorm([], [], []);) 2025.09.25 21:09:54.263278 [ 2707 ] {802de624-4ea2-48a8-a4a0-906029a89b2d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2SquaredNorm doesn't match: passed 3, should be 1: In scope SELECT L2SquaredNorm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:54.307855 [ 2707 ] {75689d5d-ed06-45d3-9fc5-16a520735369} executeQuery: Code: 42. DB::Exception: Number of arguments for function toLastDayOfMonth doesn't match: passed 3, should be 1 or 2: In scope SELECT LAST_DAY([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT LAST_DAY([], [], []);) 2025.09.25 21:09:54.311216 [ 2707 ] {75689d5d-ed06-45d3-9fc5-16a520735369} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toLastDayOfMonth doesn't match: passed 3, should be 1 or 2: In scope SELECT LAST_DAY([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:54.373631 [ 2707 ] {59c07ca8-db33-41d2-adec-13c50d1dcb5e} executeQuery: Code: 42. DB::Exception: Number of arguments for function LinfDistance doesn't match: passed 3, should be 2: In scope SELECT LinfDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT LinfDistance([], [], []);) 2025.09.25 21:09:54.378849 [ 2707 ] {59c07ca8-db33-41d2-adec-13c50d1dcb5e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function LinfDistance doesn't match: passed 3, should be 2: In scope SELECT LinfDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:54.438547 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:54.466593 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:54.485659 [ 2707 ] {54126a81-284e-4f19-94c3-9e1ca20f0bb9} executeQuery: Code: 42. DB::Exception: Number of arguments for function LinfNorm doesn't match: passed 3, should be 1: In scope SELECT LinfNorm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT LinfNorm([], [], []);) 2025.09.25 21:09:54.490436 [ 2707 ] {54126a81-284e-4f19-94c3-9e1ca20f0bb9} TCPHandler: Code: 42. DB::Exception: Number of arguments for function LinfNorm doesn't match: passed 3, should be 1: In scope SELECT LinfNorm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:54.703859 [ 2707 ] {5998e9c2-67f9-4d3a-a88a-9d133664d95f} executeQuery: Code: 42. DB::Exception: Number of arguments for function LinfNormalize doesn't match: passed 3, should be 1: In scope SELECT LinfNormalize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT LinfNormalize([], [], []);) 2025.09.25 21:09:54.708320 [ 2707 ] {5998e9c2-67f9-4d3a-a88a-9d133664d95f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function LinfNormalize doesn't match: passed 3, should be 1: In scope SELECT LinfNormalize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:54.771689 [ 2707 ] {5b35cb9c-ea41-45c1-9f01-cca2a12db3f6} executeQuery: Code: 43. DB::Exception: Arguments of function arrayLpDistance has nested type Nothing. Supported types: UInt8, UInt16, UInt32, UInt64, Int8, Int16, Int32, Int64, Float32, Float64.: In scope SELECT LpDistance([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT LpDistance([], [], []);) 2025.09.25 21:09:54.788100 [ 2707 ] {5b35cb9c-ea41-45c1-9f01-cca2a12db3f6} TCPHandler: Code: 43. DB::Exception: Arguments of function arrayLpDistance has nested type Nothing. Supported types: UInt8, UInt16, UInt32, UInt64, Int8, Int16, Int32, Int64, Float32, Float64.: In scope SELECT LpDistance([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:54.847224 [ 2707 ] {fe3350e4-cba1-4521-9783-678ac9ddcc9f} executeQuery: Code: 42. DB::Exception: Number of arguments for function LpNorm doesn't match: passed 3, should be 2: In scope SELECT LpNorm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT LpNorm([], [], []);) 2025.09.25 21:09:54.854520 [ 2707 ] {fe3350e4-cba1-4521-9783-678ac9ddcc9f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function LpNorm doesn't match: passed 3, should be 2: In scope SELECT LpNorm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:54.918889 [ 2707 ] {79a429b8-1ce1-49d3-bd44-a74c456a6906} executeQuery: Code: 42. DB::Exception: Number of arguments for function LpNormalize doesn't match: passed 3, should be 2: In scope SELECT LpNormalize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT LpNormalize([], [], []);) 2025.09.25 21:09:54.922879 [ 2707 ] {79a429b8-1ce1-49d3-bd44-a74c456a6906} TCPHandler: Code: 42. DB::Exception: Number of arguments for function LpNormalize doesn't match: passed 3, should be 2: In scope SELECT LpNormalize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:54.990664 [ 2707 ] {657ee53a-3703-4788-81cb-3334ad84eda7} executeQuery: Code: 42. DB::Exception: Number of arguments for function MACNumToString doesn't match: passed 3, should be 1: In scope SELECT MACNumToString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT MACNumToString([], [], []);) 2025.09.25 21:09:54.993100 [ 2707 ] {657ee53a-3703-4788-81cb-3334ad84eda7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function MACNumToString doesn't match: passed 3, should be 1: In scope SELECT MACNumToString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:55.133424 [ 2707 ] {2b418974-b928-439f-8074-935147c6922b} executeQuery: Code: 42. DB::Exception: Number of arguments for function MACStringToNum doesn't match: passed 3, should be 1: In scope SELECT MACStringToNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT MACStringToNum([], [], []);) 2025.09.25 21:09:55.143097 [ 2707 ] {2b418974-b928-439f-8074-935147c6922b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function MACStringToNum doesn't match: passed 3, should be 1: In scope SELECT MACStringToNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:55.154241 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:55.178209 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:55.189933 [ 2707 ] {032026ae-6224-4e06-ac16-9530b7b4bff5} executeQuery: Code: 42. DB::Exception: Number of arguments for function MACStringToOUI doesn't match: passed 3, should be 1: In scope SELECT MACStringToOUI([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT MACStringToOUI([], [], []);) 2025.09.25 21:09:55.192438 [ 2707 ] {032026ae-6224-4e06-ac16-9530b7b4bff5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function MACStringToOUI doesn't match: passed 3, should be 1: In scope SELECT MACStringToOUI([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:55.243027 [ 2707 ] {220b57d6-a411-4a61-a2c8-acb78d3722d2} executeQuery: Code: 42. DB::Exception: Number of arguments for function mapFromArrays doesn't match: passed 3, should be 2: In scope SELECT MAP_FROM_ARRAYS([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT MAP_FROM_ARRAYS([], [], []);) 2025.09.25 21:09:55.246348 [ 2707 ] {220b57d6-a411-4a61-a2c8-acb78d3722d2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function mapFromArrays doesn't match: passed 3, should be 2: In scope SELECT MAP_FROM_ARRAYS([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:55.283895 [ 2707 ] {30bedb7e-578b-41f7-8e49-b9695a5413cc} executeQuery: Code: 42. DB::Exception: Number of arguments for function MD4 doesn't match: passed 3, should be 1: In scope SELECT MD4([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT MD4([], [], []);) 2025.09.25 21:09:55.287065 [ 2707 ] {30bedb7e-578b-41f7-8e49-b9695a5413cc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function MD4 doesn't match: passed 3, should be 1: In scope SELECT MD4([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:55.340445 [ 2707 ] {52a20543-b6e0-4538-b734-81ef295f73fb} executeQuery: Code: 42. DB::Exception: Number of arguments for function MD5 doesn't match: passed 3, should be 1: In scope SELECT MD5([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT MD5([], [], []);) 2025.09.25 21:09:55.347378 [ 2707 ] {52a20543-b6e0-4538-b734-81ef295f73fb} TCPHandler: Code: 42. DB::Exception: Number of arguments for function MD5 doesn't match: passed 3, should be 1: In scope SELECT MD5([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:55.392835 [ 2707 ] {0d3ed482-db69-4acf-bb69-a343beab2dbb} executeQuery: Code: 42. DB::Exception: Number of arguments for function toMillisecond doesn't match: passed 3, should be 1 or 2: In scope SELECT MILLISECOND([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT MILLISECOND([], [], []);) 2025.09.25 21:09:55.396071 [ 2707 ] {0d3ed482-db69-4acf-bb69-a343beab2dbb} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toMillisecond doesn't match: passed 3, should be 1 or 2: In scope SELECT MILLISECOND([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:55.460025 [ 2707 ] {85074eaa-2ac0-430e-920d-36e950f61b19} executeQuery: Code: 42. DB::Exception: Number of arguments for function toMinute doesn't match: passed 3, should be 1 or 2: In scope SELECT MINUTE([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT MINUTE([], [], []);) 2025.09.25 21:09:55.463072 [ 2707 ] {85074eaa-2ac0-430e-920d-36e950f61b19} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toMinute doesn't match: passed 3, should be 1 or 2: In scope SELECT MINUTE([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:55.490035 [ 2707 ] {e49a0c0b-638a-46a1-a1b7-f9b97fe6cd97} executeQuery: Code: 42. DB::Exception: Number of arguments for function toMonth doesn't match: passed 3, should be 1 or 2: In scope SELECT MONTH([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT MONTH([], [], []);) 2025.09.25 21:09:55.502838 [ 2707 ] {e49a0c0b-638a-46a1-a1b7-f9b97fe6cd97} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toMonth doesn't match: passed 3, should be 1 or 2: In scope SELECT MONTH([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:55.639826 [ 2707 ] {4fd8c275-64f5-44c4-a2b0-199107ac20fc} executeQuery: Code: 42. DB::Exception: Number of arguments for function length doesn't match: passed 3, should be 1: In scope SELECT OCTET_LENGTH([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT OCTET_LENGTH([], [], []);) 2025.09.25 21:09:55.641892 [ 2707 ] {4fd8c275-64f5-44c4-a2b0-199107ac20fc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function length doesn't match: passed 3, should be 1: In scope SELECT OCTET_LENGTH([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:55.706878 [ 2707 ] {aa50384a-74db-4179-91fc-632a924603c6} executeQuery: Code: 42. DB::Exception: Number of arguments for function toQuarter doesn't match: passed 3, should be 1 or 2: In scope SELECT QUARTER([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT QUARTER([], [], []);) 2025.09.25 21:09:55.717522 [ 2707 ] {aa50384a-74db-4179-91fc-632a924603c6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toQuarter doesn't match: passed 3, should be 1 or 2: In scope SELECT QUARTER([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:55.768675 [ 2707 ] {74083855-8f2e-4d06-aa76-a4c37f3a370c} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'regexpExtract'. Expected: String, got: Array(Nothing): In scope SELECT REGEXP_EXTRACT([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT REGEXP_EXTRACT([], [], []);) 2025.09.25 21:09:55.772778 [ 2707 ] {74083855-8f2e-4d06-aa76-a4c37f3a370c} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'regexpExtract'. Expected: String, got: Array(Nothing): In scope SELECT REGEXP_EXTRACT([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:55.853287 [ 2707 ] {85b4972b-18fe-4323-9c89-809323017145} executeQuery: Code: 42. DB::Exception: Number of arguments for function match doesn't match: passed 3, should be 2: In scope SELECT REGEXP_MATCHES([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT REGEXP_MATCHES([], [], []);) 2025.09.25 21:09:55.863652 [ 2707 ] {85b4972b-18fe-4323-9c89-809323017145} TCPHandler: Code: 42. DB::Exception: Number of arguments for function match doesn't match: passed 3, should be 2: In scope SELECT REGEXP_MATCHES([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:55.913976 [ 2707 ] {b608dcd6-6b74-46fc-acde-efc9a2d34b94} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'replaceRegexpAll'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT REGEXP_REPLACE([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT REGEXP_REPLACE([], [], []);) 2025.09.25 21:09:55.916722 [ 2707 ] {b608dcd6-6b74-46fc-acde-efc9a2d34b94} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'replaceRegexpAll'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT REGEXP_REPLACE([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:56.040291 [ 2707 ] {1f126bea-9b2f-4555-a02c-2ecd00a6b444} executeQuery: Code: 42. DB::Exception: Number of arguments for function currentDatabase doesn't match: passed 3, should be 0: In scope SELECT SCHEMA([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT SCHEMA([], [], []);) 2025.09.25 21:09:56.042367 [ 2707 ] {1f126bea-9b2f-4555-a02c-2ecd00a6b444} TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentDatabase doesn't match: passed 3, should be 0: In scope SELECT SCHEMA([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:56.085916 [ 7178 ] {d8562b95-1a99-4007-83bd-fac0216fd860} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34794) (comment: 00089_position_functions_with_non_constant_arg.sql) (in query: SELECT count() FROM test.hits WHERE positionCaseInsensitive(URL, 'metrika') != positionCaseInsensitive(URL, materialize('metrika'));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:56.093630 [ 7178 ] {d8562b95-1a99-4007-83bd-fac0216fd860} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:56.130578 [ 3619 ] {b75354d0-81c2-42e6-9d1f-db4c9869ed8f} DatabaseReplicated (test_5): It's not recommended to explicitly specify zookeeper_path and replica_name in ReplicatedMergeTree arguments 2025.09.25 21:09:56.132400 [ 2707 ] {e7f2e59e-21ee-4008-a3d2-e84b0e03724f} executeQuery: Code: 42. DB::Exception: Number of arguments for function toSecond doesn't match: passed 3, should be 1 or 2: In scope SELECT SECOND([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT SECOND([], [], []);) 2025.09.25 21:09:56.134938 [ 3619 ] {b75354d0-81c2-42e6-9d1f-db4c9869ed8f} executeQuery: Code: 80. DB::Exception: Explicit zookeeper_path and replica_name are specified in ReplicatedMergeTree arguments. If you really want to specify it explicitly, then you should use some macros to distinguish different shards and replicas. (INCORRECT_QUERY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34802) (comment: 02477_projection_materialize_and_zero_copy.sql) (in query: CREATE TABLE t (`c1` Int64, `c2` String, `c3` DateTime, `c4` Int8, `c5` String, `c6` String, `c7` String, `c8` String, `c9` String, `c10` String, `c11` String, `c12` String, `c13` Int8, `c14` Int64, `c15` String, `c16` String, `c17` String, `c18` Int64, `c19` Int64, `c20` Int64) ENGINE = ReplicatedMergeTree('/clickhouse/test/{database}/test_02477', '1') ORDER BY c18 SETTINGS allow_remote_fs_zero_copy_replication = 1, index_granularity = 8192, index_granularity_bytes = '10Mi', min_bytes_for_wide_part = 1073741824, ratio_of_defaults_for_sparse_serialization = 1., replace_long_file_name_to_hash = false, max_file_name_length = 128, min_bytes_for_full_part_storage = 536870912, compact_parts_max_bytes_to_buffer = 71167924, compact_parts_max_granules_to_buffer = 98, compact_parts_merge_max_bytes_to_prefetch_part = 28436806, merge_max_block_size = 5907, old_parts_lifetime = 480., prefer_fetch_merged_part_size_threshold = 10737418240, vertical_merge_algorithm_min_rows_to_activate = 1000000, vertical_merge_algorithm_min_columns_to_activate = 1, min_merge_bytes_to_use_direct_io = 6730437597, concurrent_part_removal_threshold = 0, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = false, marks_compress_block_size = 67705, primary_key_compress_block_size = 76498), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:878: DB::DatabaseReplicated::checkTableEngine(DB::ASTCreateQuery const&, DB::ASTStorage&, std::shared_ptr) const @ 0x0000000027d5afc7 5. ./build_docker/./src/Databases/DatabaseReplicated.cpp:957: DB::DatabaseReplicated::checkQueryValid(std::shared_ptr const&, std::shared_ptr) const @ 0x0000000027d5e336 6. ./build_docker/./src/Databases/DatabaseReplicated.cpp:999: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5f0de 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:56.135610 [ 2707 ] {e7f2e59e-21ee-4008-a3d2-e84b0e03724f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toSecond doesn't match: passed 3, should be 1 or 2: In scope SELECT SECOND([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:56.142674 [ 3619 ] {b75354d0-81c2-42e6-9d1f-db4c9869ed8f} TCPHandler: Code: 80. DB::Exception: Explicit zookeeper_path and replica_name are specified in ReplicatedMergeTree arguments. If you really want to specify it explicitly, then you should use some macros to distinguish different shards and replicas. (INCORRECT_QUERY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:878: DB::DatabaseReplicated::checkTableEngine(DB::ASTCreateQuery const&, DB::ASTStorage&, std::shared_ptr) const @ 0x0000000027d5afc7 5. ./build_docker/./src/Databases/DatabaseReplicated.cpp:957: DB::DatabaseReplicated::checkQueryValid(std::shared_ptr const&, std::shared_ptr) const @ 0x0000000027d5e336 6. ./build_docker/./src/Databases/DatabaseReplicated.cpp:999: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5f0de 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:56.213771 [ 2707 ] {37356d14-ab16-41f9-ac5a-7fba2ae24154} executeQuery: Code: 42. DB::Exception: Number of arguments for function SHA1 doesn't match: passed 3, should be 1: In scope SELECT SHA1([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT SHA1([], [], []);) 2025.09.25 21:09:56.217815 [ 2707 ] {37356d14-ab16-41f9-ac5a-7fba2ae24154} TCPHandler: Code: 42. DB::Exception: Number of arguments for function SHA1 doesn't match: passed 3, should be 1: In scope SELECT SHA1([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:56.220686 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:56.249340 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:56.288807 [ 2707 ] {28883515-d85f-436e-bebf-948f1341ddb9} executeQuery: Code: 42. DB::Exception: Number of arguments for function SHA224 doesn't match: passed 3, should be 1: In scope SELECT SHA224([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT SHA224([], [], []);) 2025.09.25 21:09:56.301406 [ 2707 ] {28883515-d85f-436e-bebf-948f1341ddb9} TCPHandler: Code: 42. DB::Exception: Number of arguments for function SHA224 doesn't match: passed 3, should be 1: In scope SELECT SHA224([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:56.361028 [ 2707 ] {8e66e40f-772a-489f-a088-405f5e658553} executeQuery: Code: 42. DB::Exception: Number of arguments for function SHA256 doesn't match: passed 3, should be 1: In scope SELECT SHA256([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT SHA256([], [], []);) 2025.09.25 21:09:56.367305 [ 2707 ] {8e66e40f-772a-489f-a088-405f5e658553} TCPHandler: Code: 42. DB::Exception: Number of arguments for function SHA256 doesn't match: passed 3, should be 1: In scope SELECT SHA256([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:56.423201 [ 2707 ] {dca51787-7759-4078-9567-d4023cb10953} executeQuery: Code: 42. DB::Exception: Number of arguments for function SHA384 doesn't match: passed 3, should be 1: In scope SELECT SHA384([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT SHA384([], [], []);) 2025.09.25 21:09:56.433990 [ 2707 ] {dca51787-7759-4078-9567-d4023cb10953} TCPHandler: Code: 42. DB::Exception: Number of arguments for function SHA384 doesn't match: passed 3, should be 1: In scope SELECT SHA384([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:56.537794 [ 2707 ] {37103c22-8201-4f58-a54e-5b644d738f7b} executeQuery: Code: 42. DB::Exception: Number of arguments for function SHA512 doesn't match: passed 3, should be 1: In scope SELECT SHA512([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT SHA512([], [], []);) 2025.09.25 21:09:56.555682 [ 2707 ] {37103c22-8201-4f58-a54e-5b644d738f7b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function SHA512 doesn't match: passed 3, should be 1: In scope SELECT SHA512([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:56.659432 [ 2707 ] {ba3ccbfc-787b-427f-82d0-3b65da25c2cf} executeQuery: Code: 42. DB::Exception: Number of arguments for function SHA512_256 doesn't match: passed 3, should be 1: In scope SELECT SHA512_256([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT SHA512_256([], [], []);) 2025.09.25 21:09:56.666136 [ 2707 ] {ba3ccbfc-787b-427f-82d0-3b65da25c2cf} TCPHandler: Code: 42. DB::Exception: Number of arguments for function SHA512_256 doesn't match: passed 3, should be 1: In scope SELECT SHA512_256([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:56.695101 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:56.715519 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:56.735931 [ 2707 ] {5af427a2-3b6b-40d0-b8fa-c0a793883272} executeQuery: Code: 42. DB::Exception: Aggregate function stddevPop requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT STD([], [], []);) 2025.09.25 21:09:56.739934 [ 2707 ] {5af427a2-3b6b-40d0-b8fa-c0a793883272} TCPHandler: Code: 42. DB::Exception: Aggregate function stddevPop requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:56.777397 [ 2707 ] {38bf81c8-bc89-4d15-a2ea-cf461f23d4f2} executeQuery: Code: 42. DB::Exception: Aggregate function stddevPop requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT STDDEV_POP([], [], []);) 2025.09.25 21:09:56.781558 [ 2707 ] {38bf81c8-bc89-4d15-a2ea-cf461f23d4f2} TCPHandler: Code: 42. DB::Exception: Aggregate function stddevPop requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:56.812602 [ 3619 ] {44a44276-8b5f-4288-9421-45ec903aa081} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=751, jobs=728). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:40636) (comment: 01699_timezoneOffset.sql) (in query: /* Test the DST(daylight saving time) offset changing boundary*/ SELECT 'DST boundary test for Europe/Moscow:';), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:56.815391 [ 3619 ] {44a44276-8b5f-4288-9421-45ec903aa081} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=751, jobs=728). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:56.851991 [ 2707 ] {aeb4333e-7003-44f2-95b5-bad20f373ae6} executeQuery: Code: 42. DB::Exception: Aggregate function stddevSamp requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT STDDEV_SAMP([], [], []);) 2025.09.25 21:09:56.860701 [ 2707 ] {aeb4333e-7003-44f2-95b5-bad20f373ae6} TCPHandler: Code: 42. DB::Exception: Aggregate function stddevSamp requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:56.917439 [ 2707 ] {49e60e82-f0ab-4711-b206-db3ecaa89e6a} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function substringIndex, String expected: In scope SELECT SUBSTRING_INDEX([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT SUBSTRING_INDEX([], [], []);) 2025.09.25 21:09:56.920460 [ 2707 ] {49e60e82-f0ab-4711-b206-db3ecaa89e6a} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function substringIndex, String expected: In scope SELECT SUBSTRING_INDEX([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:56.998128 [ 2707 ] {ec887daf-df7d-439c-a088-1d19bfa8c201} executeQuery: Code: 42. DB::Exception: Incorrect number of arguments: expected 1 or 2 arguments: In scope SELECT SVG([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT SVG([], [], []);) 2025.09.25 21:09:57.004476 [ 2707 ] {ec887daf-df7d-439c-a088-1d19bfa8c201} TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments: expected 1 or 2 arguments: In scope SELECT SVG([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:57.049988 [ 3619 ] {b315d7f5-86e4-412e-aca8-d837ff17c936} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:40736) (comment: 00020_distinct_order_by_distributed.sql) (in query: SELECT DISTINCT `__table1`.`PredLastVisit` AS `x` FROM `test`.`visits` AS `__table1` ORDER BY `__table1`.`VisitID` ASC), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:57.049722 [ 3682 ] {b42009ae-8299-4354-bd08-63285f8ea441} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:34986) (comment: 00020_distinct_order_by_distributed.sql) (in query: SELECT DISTINCT `__table1`.`PredLastVisit` AS `x` FROM `test`.`visits` AS `__table1` ORDER BY `__table1`.`VisitID` ASC), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:57.053599 [ 3682 ] {b42009ae-8299-4354-bd08-63285f8ea441} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:57.058714 [ 3619 ] {b315d7f5-86e4-412e-aca8-d837ff17c936} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:57.073789 [ 7178 ] {b0368a2d-da1e-493d-9932-a07d58f8d4d6} executeQuery: Code: 439. DB::Exception: Received from 127.0.0.1:9000. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=1). Stack trace: 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 : While executing Remote. (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:40634) (comment: 00020_distinct_order_by_distributed.sql) (in query: SELECT count() FROM (SELECT DISTINCT PredLastVisit AS x FROM remote('127.0.0.{1,2}', test, visits) ORDER BY VisitID);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001aeb974e 3. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000002cee6c6a 4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x000000002cef780e 5. ./build_docker/./src/Client/PacketReceiver.cpp:0: DB::PacketReceiver::Task::run(std::function, std::function) @ 0x000000002cf9767e 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278ab436 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278aaa44 8. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func>>(unsigned int, unsigned int) @ 0x00000000278a8394 2025.09.25 21:09:57.078041 [ 7178 ] {b0368a2d-da1e-493d-9932-a07d58f8d4d6} TCPHandler: Code: 439. DB::Exception: Received from 127.0.0.1:9000. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=1). Stack trace: 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 : While executing Remote. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001aeb974e 3. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000002cee6c6a 4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x000000002cef780e 5. ./build_docker/./src/Client/PacketReceiver.cpp:0: DB::PacketReceiver::Task::run(std::function, std::function) @ 0x000000002cf9767e 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278ab436 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278aaa44 8. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func>>(unsigned int, unsigned int) @ 0x00000000278a8394 2025.09.25 21:09:57.097563 [ 3619 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::ffff:127.0.0.1]:40736, local: [::ffff:127.0.0.2]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:57.114260 [ 2707 ] {69f742f8-6ce6-4c8f-bf00-8e8f1c7ff3ce} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'unit' to function 'dateDiff'. Expected: String, got: Array(Nothing): In scope SELECT dateDiff([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT TIMESTAMP_DIFF([], [], []);) 2025.09.25 21:09:57.116204 [ 2707 ] {69f742f8-6ce6-4c8f-bf00-8e8f1c7ff3ce} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'unit' to function 'dateDiff'. Expected: String, got: Array(Nothing): In scope SELECT dateDiff([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:57.172693 [ 2707 ] {e5581549-67b8-4bd0-b80d-3d343479346f} executeQuery: Code: 42. DB::Exception: Number of arguments for function base64Encode doesn't match: passed 3, should be 1: In scope SELECT TO_BASE64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT TO_BASE64([], [], []);) 2025.09.25 21:09:57.177307 [ 2707 ] {e5581549-67b8-4bd0-b80d-3d343479346f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function base64Encode doesn't match: passed 3, should be 1: In scope SELECT TO_BASE64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:57.238319 [ 2707 ] {10d5c8e9-7813-4538-a12e-33fcbf3bdb8f} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDaysSinceYearZero doesn't match: passed 3, should be 1 or 2: In scope SELECT TO_DAYS([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT TO_DAYS([], [], []);) 2025.09.25 21:09:57.256536 [ 2707 ] {10d5c8e9-7813-4538-a12e-33fcbf3bdb8f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDaysSinceYearZero doesn't match: passed 3, should be 1 or 2: In scope SELECT TO_DAYS([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:57.351330 [ 2707 ] {742ff183-e739-4ff2-b3e2-963ecde37ab5} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'time' to function 'parseDateTime'. Expected: String, got: Array(Nothing): In scope SELECT TO_UNIXTIME([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT TO_UNIXTIME([], [], []);) 2025.09.25 21:09:57.354644 [ 2707 ] {742ff183-e739-4ff2-b3e2-963ecde37ab5} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'time' to function 'parseDateTime'. Expected: String, got: Array(Nothing): In scope SELECT TO_UNIXTIME([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:57.407746 [ 2707 ] {087d33ca-7c08-46be-8acb-f9cc40461699} executeQuery: Code: 42. DB::Exception: Wrong number of arguments for function ULIDStringToDateTime: should be 1 or 2: In scope SELECT ULIDStringToDateTime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ULIDStringToDateTime([], [], []);) 2025.09.25 21:09:57.413026 [ 2707 ] {087d33ca-7c08-46be-8acb-f9cc40461699} TCPHandler: Code: 42. DB::Exception: Wrong number of arguments for function ULIDStringToDateTime: should be 1 or 2: In scope SELECT ULIDStringToDateTime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:57.467311 [ 2707 ] {06f4cac1-a3ca-4211-8431-175af86bc8bd} executeQuery: Code: 42. DB::Exception: Number of arguments for function URLHash doesn't match: passed 3, should be 1 or 2.: In scope SELECT URLHash([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT URLHash([], [], []);) 2025.09.25 21:09:57.471389 [ 2707 ] {06f4cac1-a3ca-4211-8431-175af86bc8bd} TCPHandler: Code: 42. DB::Exception: Number of arguments for function URLHash doesn't match: passed 3, should be 1 or 2.: In scope SELECT URLHash([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:57.624215 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:57.632848 [ 2707 ] {fb36dcfe-2a73-4d81-a055-701aaad9aa4e} executeQuery: Code: 42. DB::Exception: Number of arguments for function URLHierarchy doesn't match: passed 3, should be 1: In scope SELECT URLHierarchy([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT URLHierarchy([], [], []);) 2025.09.25 21:09:57.635307 [ 2707 ] {fb36dcfe-2a73-4d81-a055-701aaad9aa4e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function URLHierarchy doesn't match: passed 3, should be 1: In scope SELECT URLHierarchy([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:57.646204 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:57.772835 [ 2707 ] {f05a7a4b-03ca-4415-b035-980ff0471b60} executeQuery: Code: 42. DB::Exception: Number of arguments for function URLPathHierarchy doesn't match: passed 3, should be 1: In scope SELECT URLPathHierarchy([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT URLPathHierarchy([], [], []);) 2025.09.25 21:09:57.786616 [ 2707 ] {f05a7a4b-03ca-4415-b035-980ff0471b60} TCPHandler: Code: 42. DB::Exception: Number of arguments for function URLPathHierarchy doesn't match: passed 3, should be 1: In scope SELECT URLPathHierarchy([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:57.833478 [ 2707 ] {3506f93f-1ea6-495d-a2db-92c85e38eef1} executeQuery: Code: 42. DB::Exception: Number of arguments for function UTCTimestamp doesn't match: passed 3, should be 0: In scope SELECT UTCTimestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT UTCTimestamp([], [], []);) 2025.09.25 21:09:57.839758 [ 2707 ] {3506f93f-1ea6-495d-a2db-92c85e38eef1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function UTCTimestamp doesn't match: passed 3, should be 0: In scope SELECT UTCTimestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:57.881551 [ 2707 ] {69b57be3-6f42-4bb8-a0a3-0ed9bf3ca49a} executeQuery: Code: 42. DB::Exception: Number of arguments for function UTCTimestamp doesn't match: passed 3, should be 0: In scope SELECT UTC_timestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT UTC_timestamp([], [], []);) 2025.09.25 21:09:57.885568 [ 2707 ] {69b57be3-6f42-4bb8-a0a3-0ed9bf3ca49a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function UTCTimestamp doesn't match: passed 3, should be 0: In scope SELECT UTC_timestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:57.999057 [ 2707 ] {9527f764-0715-4c4e-aaa9-63e2e0678cfb} executeQuery: Code: 42. DB::Exception: Number of arguments for function UUIDNumToString doesn't match: passed 3, should be 1 or 2: In scope SELECT UUIDNumToString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT UUIDNumToString([], [], []);) 2025.09.25 21:09:58.002974 [ 2707 ] {9527f764-0715-4c4e-aaa9-63e2e0678cfb} TCPHandler: Code: 42. DB::Exception: Number of arguments for function UUIDNumToString doesn't match: passed 3, should be 1 or 2: In scope SELECT UUIDNumToString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:58.045793 [ 2707 ] {99c053b5-f96c-4a98-b191-d46de1f921ef} executeQuery: Code: 42. DB::Exception: Number of arguments for function UUIDStringToNum doesn't match: passed 3, should be 1 or 2: In scope SELECT UUIDStringToNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT UUIDStringToNum([], [], []);) 2025.09.25 21:09:58.049198 [ 2707 ] {99c053b5-f96c-4a98-b191-d46de1f921ef} TCPHandler: Code: 42. DB::Exception: Number of arguments for function UUIDStringToNum doesn't match: passed 3, should be 1 or 2: In scope SELECT UUIDStringToNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:58.085316 [ 2707 ] {77b11d98-8aae-4f37-9c37-0fd095060c9a} executeQuery: Code: 42. DB::Exception: Number of arguments for function UUIDToNum doesn't match: passed 3, should be 1 or 2: In scope SELECT UUIDToNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT UUIDToNum([], [], []);) 2025.09.25 21:09:58.088828 [ 2707 ] {77b11d98-8aae-4f37-9c37-0fd095060c9a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function UUIDToNum doesn't match: passed 3, should be 1 or 2: In scope SELECT UUIDToNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:58.134972 [ 2707 ] {b4a456b3-9739-414d-83dc-d472a980206f} executeQuery: Code: 42. DB::Exception: Wrong number of arguments for function UUIDv7ToDateTime: should be 1 or 2: In scope SELECT UUIDv7ToDateTime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT UUIDv7ToDateTime([], [], []);) 2025.09.25 21:09:58.139121 [ 2707 ] {b4a456b3-9739-414d-83dc-d472a980206f} TCPHandler: Code: 42. DB::Exception: Wrong number of arguments for function UUIDv7ToDateTime: should be 1 or 2: In scope SELECT UUIDv7ToDateTime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:58.204902 [ 2707 ] {176e46ec-2572-48ee-a979-fe08f1e5e20e} executeQuery: Code: 42. DB::Exception: Aggregate function varPop requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT VAR_POP([], [], []);) 2025.09.25 21:09:58.210520 [ 2707 ] {176e46ec-2572-48ee-a979-fe08f1e5e20e} TCPHandler: Code: 42. DB::Exception: Aggregate function varPop requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:58.275937 [ 2707 ] {59759342-1595-49e3-9d69-3ec19e725f36} executeQuery: Code: 42. DB::Exception: Aggregate function varSamp requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT VAR_SAMP([], [], []);) 2025.09.25 21:09:58.283239 [ 2707 ] {59759342-1595-49e3-9d69-3ec19e725f36} TCPHandler: Code: 42. DB::Exception: Aggregate function varSamp requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:58.352846 [ 2707 ] {2528ee81-b3f7-450a-acde-bc3991b885c7} executeQuery: Code: 42. DB::Exception: Number of arguments for function toYear doesn't match: passed 3, should be 1 or 2: In scope SELECT YEAR([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT YEAR([], [], []);) 2025.09.25 21:09:58.358494 [ 2707 ] {2528ee81-b3f7-450a-acde-bc3991b885c7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toYear doesn't match: passed 3, should be 1 or 2: In scope SELECT YEAR([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:58.422560 [ 2707 ] {b9315d23-68f4-47b5-81db-e20f1946acdd} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'YYYYMMDDToDate'. Expected 1 argument, got 3 arguments: In scope SELECT YYYYMMDDToDate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT YYYYMMDDToDate([], [], []);) 2025.09.25 21:09:58.442272 [ 2707 ] {b9315d23-68f4-47b5-81db-e20f1946acdd} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'YYYYMMDDToDate'. Expected 1 argument, got 3 arguments: In scope SELECT YYYYMMDDToDate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:58.535338 [ 2707 ] {7a23fb94-4020-4921-b5a0-13202a2f8bb5} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'YYYYMMDDToDate32'. Expected 1 argument, got 3 arguments: In scope SELECT YYYYMMDDToDate32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT YYYYMMDDToDate32([], [], []);) 2025.09.25 21:09:58.546954 [ 2707 ] {7a23fb94-4020-4921-b5a0-13202a2f8bb5} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'YYYYMMDDToDate32'. Expected 1 argument, got 3 arguments: In scope SELECT YYYYMMDDToDate32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:58.630206 [ 2707 ] {996b9bad-829d-4598-83fc-7000b90176b0} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'YYYYMMDDhhmmssToDateTime'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT YYYYMMDDhhmmssToDateTime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT YYYYMMDDhhmmssToDateTime([], [], []);) 2025.09.25 21:09:58.639279 [ 2707 ] {996b9bad-829d-4598-83fc-7000b90176b0} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'YYYYMMDDhhmmssToDateTime'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT YYYYMMDDhhmmssToDateTime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:58.691564 [ 2707 ] {b4a27a55-922a-4443-b92c-ad91a2daab20} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'YYYYMMDDhhmmss' to function 'YYYYMMDDhhmmssToDateTime64'. Expected: Number, got: Array(Nothing): In scope SELECT YYYYMMDDhhmmssToDateTime64([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT YYYYMMDDhhmmssToDateTime64([], [], []);) 2025.09.25 21:09:58.697880 [ 2707 ] {b4a27a55-922a-4443-b92c-ad91a2daab20} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'YYYYMMDDhhmmss' to function 'YYYYMMDDhhmmssToDateTime64'. Expected: Number, got: Array(Nothing): In scope SELECT YYYYMMDDhhmmssToDateTime64([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:58.762506 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:58.774206 [ 2707 ] {88b2b3ef-165d-4432-86dd-93b7723536ee} executeQuery: Code: 42. DB::Exception: Number of arguments for function _CAST doesn't match: passed 3, should be 2: In scope SELECT _CAST([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT _CAST([], [], []);) 2025.09.25 21:09:58.776232 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:09:58.780526 [ 2707 ] {88b2b3ef-165d-4432-86dd-93b7723536ee} TCPHandler: Code: 42. DB::Exception: Number of arguments for function _CAST doesn't match: passed 3, should be 2: In scope SELECT _CAST([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:58.861319 [ 2707 ] {f7f98607-5266-46c9-a086-d0879a8cb710} executeQuery: Code: 42. DB::Exception: Number of arguments for function __actionName doesn't match: passed 3, should be 2: In scope SELECT __actionName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT __actionName([], [], []);) 2025.09.25 21:09:58.865503 [ 2707 ] {f7f98607-5266-46c9-a086-d0879a8cb710} TCPHandler: Code: 42. DB::Exception: Number of arguments for function __actionName doesn't match: passed 3, should be 2: In scope SELECT __actionName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:58.918297 [ 2707 ] {563a9977-6123-40d3-a786-788021a6f543} executeQuery: Code: 42. DB::Exception: Number of arguments for function __bitBoolMaskAnd doesn't match: passed 3, should be 2: In scope SELECT __bitBoolMaskAnd([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT __bitBoolMaskAnd([], [], []);) 2025.09.25 21:09:58.932211 [ 2707 ] {563a9977-6123-40d3-a786-788021a6f543} TCPHandler: Code: 42. DB::Exception: Number of arguments for function __bitBoolMaskAnd doesn't match: passed 3, should be 2: In scope SELECT __bitBoolMaskAnd([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:59.004871 [ 2707 ] {881e3582-34ea-499b-91a3-36b7c3c964d7} executeQuery: Code: 42. DB::Exception: Number of arguments for function __bitBoolMaskOr doesn't match: passed 3, should be 2: In scope SELECT __bitBoolMaskOr([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT __bitBoolMaskOr([], [], []);) 2025.09.25 21:09:59.008691 [ 2707 ] {881e3582-34ea-499b-91a3-36b7c3c964d7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function __bitBoolMaskOr doesn't match: passed 3, should be 2: In scope SELECT __bitBoolMaskOr([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:59.080302 [ 2707 ] {4672108f-6ecc-4584-ab1a-cde444b96c65} executeQuery: Code: 42. DB::Exception: Number of arguments for function __bitSwapLastTwo doesn't match: passed 3, should be 1: In scope SELECT __bitSwapLastTwo([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT __bitSwapLastTwo([], [], []);) 2025.09.25 21:09:59.089766 [ 2707 ] {4672108f-6ecc-4584-ab1a-cde444b96c65} TCPHandler: Code: 42. DB::Exception: Number of arguments for function __bitSwapLastTwo doesn't match: passed 3, should be 1: In scope SELECT __bitSwapLastTwo([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:59.254249 [ 2707 ] {14d1199e-d3d2-4175-8507-793bfeea020e} executeQuery: Code: 42. DB::Exception: Number of arguments for function __bitWrapperFunc doesn't match: passed 3, should be 1: In scope SELECT __bitWrapperFunc([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT __bitWrapperFunc([], [], []);) 2025.09.25 21:09:59.260551 [ 2707 ] {14d1199e-d3d2-4175-8507-793bfeea020e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function __bitWrapperFunc doesn't match: passed 3, should be 1: In scope SELECT __bitWrapperFunc([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:59.365679 [ 2707 ] {87f24add-a146-445d-bba0-e118e7274776} executeQuery: Code: 42. DB::Exception: Number of arguments for function __getScalar doesn't match: passed 3, should be 1: In scope SELECT __getScalar([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT __getScalar([], [], []);) 2025.09.25 21:09:59.372222 [ 2707 ] {87f24add-a146-445d-bba0-e118e7274776} TCPHandler: Code: 42. DB::Exception: Number of arguments for function __getScalar doesn't match: passed 3, should be 1: In scope SELECT __getScalar([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:59.456781 [ 2707 ] {73417b07-1d9a-4125-84dc-25c11959a291} executeQuery: Code: 42. DB::Exception: Number of arguments for function __scalarSubqueryResult doesn't match: passed 3, should be 1: In scope SELECT __scalarSubqueryResult([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT __scalarSubqueryResult([], [], []);) 2025.09.25 21:09:59.471426 [ 2707 ] {73417b07-1d9a-4125-84dc-25c11959a291} TCPHandler: Code: 42. DB::Exception: Number of arguments for function __scalarSubqueryResult doesn't match: passed 3, should be 1: In scope SELECT __scalarSubqueryResult([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:59.563487 [ 2707 ] {89649789-fe4e-49ea-81b8-a5690ef8b278} executeQuery: Code: 42. DB::Exception: Number of arguments for function abs doesn't match: passed 3, should be 1: In scope SELECT abs([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT abs([], [], []);) 2025.09.25 21:09:59.567905 [ 2707 ] {89649789-fe4e-49ea-81b8-a5690ef8b278} TCPHandler: Code: 42. DB::Exception: Number of arguments for function abs doesn't match: passed 3, should be 1: In scope SELECT abs([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:59.759988 [ 2707 ] {75c207cf-3aa2-4ecd-be87-1fc2cd36f914} executeQuery: Code: 42. DB::Exception: Number of arguments for function accurateCast doesn't match: passed 3, should be 2: In scope SELECT accurateCast([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT accurateCast([], [], []);) 2025.09.25 21:09:59.767795 [ 2707 ] {75c207cf-3aa2-4ecd-be87-1fc2cd36f914} TCPHandler: Code: 42. DB::Exception: Number of arguments for function accurateCast doesn't match: passed 3, should be 2: In scope SELECT accurateCast([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:09:59.888133 [ 2707 ] {948f3494-0383-4afd-9821-9211f801ee6e} executeQuery: Code: 43. DB::Exception: Second argument to accurateCastOrDefault must be a constant string describing type. Actual Array(Nothing): In scope SELECT accurateCastOrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT accurateCastOrDefault([], [], []);) 2025.09.25 21:09:59.901694 [ 2707 ] {948f3494-0383-4afd-9821-9211f801ee6e} TCPHandler: Code: 43. DB::Exception: Second argument to accurateCastOrDefault must be a constant string describing type. Actual Array(Nothing): In scope SELECT accurateCastOrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:09:59.926081 [ 7178 ] {9cfcb95f-248e-40a0-8ce8-d98dbd8de5bc} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:40672) (comment: 00087_where_0.sql) (in query: SELECT CounterID, uniq(UserID) FROM test.hits WHERE 0 != 0 GROUP BY CounterID SETTINGS optimize_aggregation_in_order = 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:59.933300 [ 7178 ] {9cfcb95f-248e-40a0-8ce8-d98dbd8de5bc} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:09:59.987030 [ 2707 ] {e221c9b4-8f94-48d2-9a7b-5628c73b0cda} executeQuery: Code: 42. DB::Exception: Number of arguments for function accurateCastOrNull doesn't match: passed 3, should be 2: In scope SELECT accurateCastOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT accurateCastOrNull([], [], []);) 2025.09.25 21:09:59.996155 [ 2707 ] {e221c9b4-8f94-48d2-9a7b-5628c73b0cda} TCPHandler: Code: 42. DB::Exception: Number of arguments for function accurateCastOrNull doesn't match: passed 3, should be 2: In scope SELECT accurateCastOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:00.013936 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:00.057480 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:00.126312 [ 2707 ] {f4372d86-a34b-45fc-9394-cdb89cfa633d} executeQuery: Code: 42. DB::Exception: Number of arguments for function acos doesn't match: passed 3, should be 1: In scope SELECT acos([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT acos([], [], []);) 2025.09.25 21:10:00.133118 [ 2707 ] {f4372d86-a34b-45fc-9394-cdb89cfa633d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function acos doesn't match: passed 3, should be 1: In scope SELECT acos([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:00.294835 [ 2707 ] {aadf9686-8446-43b8-a658-b9b82a8f9fc5} executeQuery: Code: 42. DB::Exception: Number of arguments for function acosh doesn't match: passed 3, should be 1: In scope SELECT acosh([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT acosh([], [], []);) 2025.09.25 21:10:00.309158 [ 2707 ] {aadf9686-8446-43b8-a658-b9b82a8f9fc5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function acosh doesn't match: passed 3, should be 1: In scope SELECT acosh([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:00.464932 [ 2707 ] {0e32df63-a89d-4395-8288-bd8ba230b276} executeQuery: Code: 42. DB::Exception: Number of arguments for function addDate doesn't match: passed 3, should be 2: In scope SELECT addDate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addDate([], [], []);) 2025.09.25 21:10:00.471939 [ 2707 ] {0e32df63-a89d-4395-8288-bd8ba230b276} TCPHandler: Code: 42. DB::Exception: Number of arguments for function addDate doesn't match: passed 3, should be 2: In scope SELECT addDate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:00.549949 [ 2707 ] {75a3df08-9269-42ec-8a13-90f70b8af5ec} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addDays. Must be a DateTime/DateTime64: In scope SELECT addDays([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addDays([], [], []);) 2025.09.25 21:10:00.568186 [ 2707 ] {75a3df08-9269-42ec-8a13-90f70b8af5ec} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addDays. Must be a DateTime/DateTime64: In scope SELECT addDays([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:00.618540 [ 2707 ] {b8e1c343-b43f-4d22-a7a9-a4cbe2557cb9} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addHours. Must be a DateTime/DateTime64: In scope SELECT addHours([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addHours([], [], []);) 2025.09.25 21:10:00.622959 [ 2707 ] {b8e1c343-b43f-4d22-a7a9-a4cbe2557cb9} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addHours. Must be a DateTime/DateTime64: In scope SELECT addHours([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:00.682055 [ 2707 ] {1cfadff5-eb68-4814-8487-0c9c207a7f1a} executeQuery: Code: 42. DB::Exception: Number of arguments for function addInterval doesn't match: passed 3, should be 2: In scope SELECT addInterval([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addInterval([], [], []);) 2025.09.25 21:10:00.687573 [ 2707 ] {1cfadff5-eb68-4814-8487-0c9c207a7f1a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function addInterval doesn't match: passed 3, should be 2: In scope SELECT addInterval([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:00.733593 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:00.756837 [ 2707 ] {6ffd3fa3-5579-4f79-a3fc-20ca173e8cd0} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addMicroseconds. Must be a DateTime/DateTime64: In scope SELECT addMicroseconds([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addMicroseconds([], [], []);) 2025.09.25 21:10:00.764770 [ 2707 ] {6ffd3fa3-5579-4f79-a3fc-20ca173e8cd0} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addMicroseconds. Must be a DateTime/DateTime64: In scope SELECT addMicroseconds([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:00.766623 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:00.887064 [ 2707 ] {3e2b38a6-6a64-45d0-bd74-7134b34a53de} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addMilliseconds. Must be a DateTime/DateTime64: In scope SELECT addMilliseconds([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addMilliseconds([], [], []);) 2025.09.25 21:10:00.900811 [ 2707 ] {3e2b38a6-6a64-45d0-bd74-7134b34a53de} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addMilliseconds. Must be a DateTime/DateTime64: In scope SELECT addMilliseconds([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:00.997402 [ 2707 ] {d8014cda-7dd9-450f-b2a2-cdf0f95cab01} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addMinutes. Must be a DateTime/DateTime64: In scope SELECT addMinutes([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addMinutes([], [], []);) 2025.09.25 21:10:01.003600 [ 2707 ] {d8014cda-7dd9-450f-b2a2-cdf0f95cab01} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addMinutes. Must be a DateTime/DateTime64: In scope SELECT addMinutes([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:01.138409 [ 2707 ] {60ddba40-a7f8-4a89-977d-3c06e566132a} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addMonths. Must be a DateTime/DateTime64: In scope SELECT addMonths([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addMonths([], [], []);) 2025.09.25 21:10:01.154201 [ 2707 ] {60ddba40-a7f8-4a89-977d-3c06e566132a} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addMonths. Must be a DateTime/DateTime64: In scope SELECT addMonths([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:01.221575 [ 2707 ] {735dc9a9-8bc1-4d95-8706-164665c61a80} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addNanoseconds. Must be a DateTime/DateTime64: In scope SELECT addNanoseconds([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addNanoseconds([], [], []);) 2025.09.25 21:10:01.316224 [ 2707 ] {735dc9a9-8bc1-4d95-8706-164665c61a80} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addNanoseconds. Must be a DateTime/DateTime64: In scope SELECT addNanoseconds([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:01.449489 [ 2707 ] {86b5963a-51c1-40e7-8499-45714b063d57} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addQuarters. Must be a DateTime/DateTime64: In scope SELECT addQuarters([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addQuarters([], [], []);) 2025.09.25 21:10:01.455649 [ 2707 ] {86b5963a-51c1-40e7-8499-45714b063d57} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addQuarters. Must be a DateTime/DateTime64: In scope SELECT addQuarters([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:01.531419 [ 2707 ] {23d37978-0916-40d6-b4fe-929c2d90fe4e} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addSeconds. Must be a DateTime/DateTime64: In scope SELECT addSeconds([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addSeconds([], [], []);) 2025.09.25 21:10:01.539124 [ 2707 ] {23d37978-0916-40d6-b4fe-929c2d90fe4e} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addSeconds. Must be a DateTime/DateTime64: In scope SELECT addSeconds([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:01.563381 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:01.587985 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:01.626223 [ 2707 ] {61bb75f7-cfdf-4d10-b90f-4df3bffdf616} executeQuery: Code: 42. DB::Exception: Number of arguments for function addTupleOfIntervals doesn't match: passed 3, should be 2: In scope SELECT addTupleOfIntervals([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addTupleOfIntervals([], [], []);) 2025.09.25 21:10:01.632921 [ 2707 ] {61bb75f7-cfdf-4d10-b90f-4df3bffdf616} TCPHandler: Code: 42. DB::Exception: Number of arguments for function addTupleOfIntervals doesn't match: passed 3, should be 2: In scope SELECT addTupleOfIntervals([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:01.689463 [ 2707 ] {924a9b9f-16d0-4397-aaf5-662e48a8846e} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addWeeks. Must be a DateTime/DateTime64: In scope SELECT addWeeks([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addWeeks([], [], []);) 2025.09.25 21:10:01.701099 [ 2707 ] {924a9b9f-16d0-4397-aaf5-662e48a8846e} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addWeeks. Must be a DateTime/DateTime64: In scope SELECT addWeeks([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:01.791039 [ 2707 ] {3bc01a65-982b-4e88-b76f-55f99e69cc80} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addYears. Must be a DateTime/DateTime64: In scope SELECT addYears([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addYears([], [], []);) 2025.09.25 21:10:01.798573 [ 2707 ] {3bc01a65-982b-4e88-b76f-55f99e69cc80} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function addYears. Must be a DateTime/DateTime64: In scope SELECT addYears([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:01.902366 [ 2707 ] {f65e7ff5-f038-41c8-bfc2-cf2d61b7a8f1} executeQuery: Code: 42. DB::Exception: Number of arguments for function addressToLine doesn't match: passed 3, should be 1: In scope SELECT addressToLine([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addressToLine([], [], []);) 2025.09.25 21:10:01.913700 [ 2707 ] {f65e7ff5-f038-41c8-bfc2-cf2d61b7a8f1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function addressToLine doesn't match: passed 3, should be 1: In scope SELECT addressToLine([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:02.004512 [ 2707 ] {49dee645-3ef9-4d29-af29-b8873f66d85a} executeQuery: Code: 42. DB::Exception: Number of arguments for function addressToLineWithInlines doesn't match: passed 3, should be 1: In scope SELECT addressToLineWithInlines([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addressToLineWithInlines([], [], []);) 2025.09.25 21:10:02.009270 [ 2707 ] {49dee645-3ef9-4d29-af29-b8873f66d85a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function addressToLineWithInlines doesn't match: passed 3, should be 1: In scope SELECT addressToLineWithInlines([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:02.091579 [ 2707 ] {ceeb8ebf-d4d9-4adc-adb1-904b51231240} executeQuery: Code: 42. DB::Exception: Number of arguments for function addressToSymbol doesn't match: passed 3, should be 1: In scope SELECT addressToSymbol([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT addressToSymbol([], [], []);) 2025.09.25 21:10:02.096500 [ 2707 ] {ceeb8ebf-d4d9-4adc-adb1-904b51231240} TCPHandler: Code: 42. DB::Exception: Number of arguments for function addressToSymbol doesn't match: passed 3, should be 1: In scope SELECT addressToSymbol([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:02.196829 [ 2707 ] {e64c1c9c-b275-45cb-8e2c-cf24d646f88b} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'mode' to function 'aes_decrypt_mysql'. Expected: decryption mode string, got: Array(Nothing): In scope SELECT aes_decrypt_mysql([], '[HIDDEN]'). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT aes_decrypt_mysql([], '[HIDDEN]')) 2025.09.25 21:10:02.201775 [ 2707 ] {e64c1c9c-b275-45cb-8e2c-cf24d646f88b} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'mode' to function 'aes_decrypt_mysql'. Expected: decryption mode string, got: Array(Nothing): In scope SELECT aes_decrypt_mysql([], '[HIDDEN]'). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:02.255951 [ 2707 ] {737b2202-ebd2-4676-bb7b-b86a3b5f683b} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'mode' to function 'aes_encrypt_mysql'. Expected: encryption mode string, got: Array(Nothing): In scope SELECT aes_encrypt_mysql([], '[HIDDEN]'). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT aes_encrypt_mysql([], '[HIDDEN]')) 2025.09.25 21:10:02.260162 [ 2707 ] {737b2202-ebd2-4676-bb7b-b86a3b5f683b} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'mode' to function 'aes_encrypt_mysql'. Expected: encryption mode string, got: Array(Nothing): In scope SELECT aes_encrypt_mysql([], '[HIDDEN]'). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:02.395609 [ 2707 ] {e487ce55-727d-4b95-a256-2903dfec8b09} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'unit' to function 'age'. Expected: String, got: Array(Nothing): In scope SELECT age([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT age([], [], []);) 2025.09.25 21:10:02.398512 [ 2707 ] {e487ce55-727d-4b95-a256-2903dfec8b09} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'unit' to function 'age'. Expected: String, got: Array(Nothing): In scope SELECT age([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:02.506130 [ 2707 ] {a245f870-f018-4022-87c9-75e962593da0} executeQuery: Code: 503. DB::Exception: Aggregate function aggThrow has thrown exception successfully: While executing AggregatingTransform. (AGGREGATE_FUNCTION_THROW) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT aggThrow([], [], []);) 2025.09.25 21:10:02.510351 [ 2707 ] {a245f870-f018-4022-87c9-75e962593da0} TCPHandler: Code: 503. DB::Exception: Aggregate function aggThrow has thrown exception successfully: While executing AggregatingTransform. (AGGREGATE_FUNCTION_THROW) 2025.09.25 21:10:02.559857 [ 2707 ] {3e2a7412-e69b-49c0-8437-02f6d1d62978} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'alphaTokens'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT alphaTokens([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT alphaTokens([], [], []);) 2025.09.25 21:10:02.563774 [ 2707 ] {3e2a7412-e69b-49c0-8437-02f6d1d62978} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'alphaTokens'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT alphaTokens([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:02.593025 [ 2707 ] {efc903c0-35eb-46b1-a429-281f5415ec86} executeQuery: Code: 42. DB::Exception: Aggregate function analysisOfVariance requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT analysisOfVariance([], [], []);) 2025.09.25 21:10:02.596914 [ 2707 ] {efc903c0-35eb-46b1-a429-281f5415ec86} TCPHandler: Code: 42. DB::Exception: Aggregate function analysisOfVariance requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:02.647822 [ 2707 ] {b5a5c5c6-df59-4016-af75-6e67a41aaa08} executeQuery: Code: 43. DB::Exception: Illegal type (Array(Nothing)) of 1 argument of function and: In scope SELECT [] AND [] AND []. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT and([], [], []);) 2025.09.25 21:10:02.651749 [ 2707 ] {b5a5c5c6-df59-4016-af75-6e67a41aaa08} TCPHandler: Code: 43. DB::Exception: Illegal type (Array(Nothing)) of 1 argument of function and: In scope SELECT [] AND [] AND []. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:02.700698 [ 2707 ] {022c3a94-5e8c-4a30-a0c9-5f02881c8310} executeQuery: Code: 42. DB::Exception: Aggregate function analysisOfVariance requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT anova([], [], []);) 2025.09.25 21:10:02.704325 [ 2707 ] {022c3a94-5e8c-4a30-a0c9-5f02881c8310} TCPHandler: Code: 42. DB::Exception: Aggregate function analysisOfVariance requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:02.756358 [ 2707 ] {cb954b10-ddb8-4fd2-8565-b986dde2517c} executeQuery: Code: 42. DB::Exception: Aggregate function any requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT any([], [], []);) 2025.09.25 21:10:02.761146 [ 2707 ] {cb954b10-ddb8-4fd2-8565-b986dde2517c} TCPHandler: Code: 42. DB::Exception: Aggregate function any requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:02.787514 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:02.801076 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:02.806698 [ 2707 ] {f9a43fd4-d6f8-4060-b05f-a3c5a02fda69} executeQuery: Code: 42. DB::Exception: Aggregate function anyHeavy requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT anyHeavy([], [], []);) 2025.09.25 21:10:02.809893 [ 2707 ] {f9a43fd4-d6f8-4060-b05f-a3c5a02fda69} TCPHandler: Code: 42. DB::Exception: Aggregate function anyHeavy requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:02.907323 [ 2707 ] {077139ce-f4bd-4e85-a61b-3ba632f213ac} executeQuery: Code: 42. DB::Exception: Aggregate function anyLast requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT anyLast([], [], []);) 2025.09.25 21:10:02.912646 [ 2707 ] {077139ce-f4bd-4e85-a61b-3ba632f213ac} TCPHandler: Code: 42. DB::Exception: Aggregate function anyLast requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:02.987951 [ 2707 ] {2541167c-18d9-4266-941d-732c9247d04b} executeQuery: Code: 42. DB::Exception: Aggregate function anyLast_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT anyLast_respect_nulls([], [], []);) 2025.09.25 21:10:03.005484 [ 2707 ] {2541167c-18d9-4266-941d-732c9247d04b} TCPHandler: Code: 42. DB::Exception: Aggregate function anyLast_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:03.056929 [ 2707 ] {0e863688-f61b-4ac7-bb1c-f877c6f7c658} executeQuery: Code: 42. DB::Exception: Aggregate function any_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT any_respect_nulls([], [], []);) 2025.09.25 21:10:03.061392 [ 2707 ] {0e863688-f61b-4ac7-bb1c-f877c6f7c658} TCPHandler: Code: 42. DB::Exception: Aggregate function any_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:03.129098 [ 2707 ] {eb6a567a-e91b-4fa7-b112-8367e3c39c38} executeQuery: Code: 42. DB::Exception: Aggregate function any requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT any_value([], [], []);) 2025.09.25 21:10:03.133564 [ 2707 ] {eb6a567a-e91b-4fa7-b112-8367e3c39c38} TCPHandler: Code: 42. DB::Exception: Aggregate function any requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:03.179092 [ 2707 ] {49415250-db08-4546-8830-e38671dce153} executeQuery: Code: 42. DB::Exception: Aggregate function any_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT any_value_respect_nulls([], [], []);) 2025.09.25 21:10:03.182938 [ 2707 ] {49415250-db08-4546-8830-e38671dce153} TCPHandler: Code: 42. DB::Exception: Aggregate function any_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:03.284549 [ 2707 ] {ed6ae910-12a6-4938-ab4b-9c9a70d3e715} executeQuery: Code: 42. DB::Exception: Number of arguments for function appendTrailingCharIfAbsent doesn't match: passed 3, should be 2: In scope SELECT appendTrailingCharIfAbsent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT appendTrailingCharIfAbsent([], [], []);) 2025.09.25 21:10:03.294719 [ 2707 ] {ed6ae910-12a6-4938-ab4b-9c9a70d3e715} TCPHandler: Code: 42. DB::Exception: Number of arguments for function appendTrailingCharIfAbsent doesn't match: passed 3, should be 2: In scope SELECT appendTrailingCharIfAbsent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:03.325420 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:03.359181 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:03.403145 [ 2707 ] {82590dde-00c4-4c30-82b7-0201f9594852} executeQuery: Code: 42. DB::Exception: Aggregate function approx_top_k requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT approx_top_count([], [], []);) 2025.09.25 21:10:03.407826 [ 2707 ] {82590dde-00c4-4c30-82b7-0201f9594852} TCPHandler: Code: 42. DB::Exception: Aggregate function approx_top_k requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:03.479530 [ 2707 ] {cc56efb0-6a05-4f66-97e5-1307c0d1e23c} executeQuery: Code: 42. DB::Exception: Aggregate function approx_top_k requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT approx_top_k([], [], []);) 2025.09.25 21:10:03.483909 [ 2707 ] {cc56efb0-6a05-4f66-97e5-1307c0d1e23c} TCPHandler: Code: 42. DB::Exception: Aggregate function approx_top_k requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:03.583704 [ 2707 ] {594502af-91ec-46ad-aa55-f4bffc4ed628} executeQuery: Code: 42. DB::Exception: Aggregate function approx_top_sum requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT approx_top_sum([], [], []);) 2025.09.25 21:10:03.590182 [ 2707 ] {594502af-91ec-46ad-aa55-f4bffc4ed628} TCPHandler: Code: 42. DB::Exception: Aggregate function approx_top_sum requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:03.662934 [ 2707 ] {0adce4d9-bfff-4aaf-8b5c-b7dac89b75e4} executeQuery: Code: 42. DB::Exception: Aggregate function argMax requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT argMax([], [], []);) 2025.09.25 21:10:03.667397 [ 2707 ] {0adce4d9-bfff-4aaf-8b5c-b7dac89b75e4} TCPHandler: Code: 42. DB::Exception: Aggregate function argMax requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:03.729010 [ 2707 ] {1fe7d22c-177d-4961-9ab8-0efaa3b5d2a3} executeQuery: Code: 42. DB::Exception: Aggregate function argMin requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT argMin([], [], []);) 2025.09.25 21:10:03.733125 [ 2707 ] {1fe7d22c-177d-4961-9ab8-0efaa3b5d2a3} TCPHandler: Code: 42. DB::Exception: Aggregate function argMin requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:03.769392 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:03.799685 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:03.952024 [ 2707 ] {2fd6b53e-c35a-4548-8b67-fd9fa6eea123} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayAUC doesn't match: passed 3, should be 2: In scope SELECT arrayAUC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayAUC([], [], []);) 2025.09.25 21:10:03.958742 [ 2707 ] {2fd6b53e-c35a-4548-8b67-fd9fa6eea123} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayAUC doesn't match: passed 3, should be 2: In scope SELECT arrayAUC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:04.005078 [ 2707 ] {94e759f6-4695-4579-b31d-d719b64aa07c} executeQuery: Code: 43. DB::Exception: First argument for function arrayAll must be a function. Actual Array(Nothing): In scope SELECT arrayAll([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayAll([], [], []);) 2025.09.25 21:10:04.009325 [ 2707 ] {94e759f6-4695-4579-b31d-d719b64aa07c} TCPHandler: Code: 43. DB::Exception: First argument for function arrayAll must be a function. Actual Array(Nothing): In scope SELECT arrayAll([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:04.065618 [ 2707 ] {1197a691-ab48-40f1-99b8-87ea16987fbe} executeQuery: Code: 43. DB::Exception: First argument for function arrayAvg must be a function. Actual Array(Nothing): In scope SELECT arrayAvg([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayAvg([], [], []);) 2025.09.25 21:10:04.069024 [ 2707 ] {1197a691-ab48-40f1-99b8-87ea16987fbe} TCPHandler: Code: 43. DB::Exception: First argument for function arrayAvg must be a function. Actual Array(Nothing): In scope SELECT arrayAvg([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:04.111143 [ 2707 ] {ec58674e-0b23-4a6a-af78-7e6f2531db90} executeQuery: Code: 43. DB::Exception: First argument for function arrayCompact must be a function. Actual Array(Nothing): In scope SELECT arrayCompact([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayCompact([], [], []);) 2025.09.25 21:10:04.121179 [ 2707 ] {ec58674e-0b23-4a6a-af78-7e6f2531db90} TCPHandler: Code: 43. DB::Exception: First argument for function arrayCompact must be a function. Actual Array(Nothing): In scope SELECT arrayCompact([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:04.269067 [ 2707 ] {a3b25a5a-0089-4b9c-a54d-c0b1948eb7bc} executeQuery: Code: 43. DB::Exception: First argument for function arrayCount must be a function. Actual Array(Nothing): In scope SELECT arrayCount([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayCount([], [], []);) 2025.09.25 21:10:04.273761 [ 2707 ] {a3b25a5a-0089-4b9c-a54d-c0b1948eb7bc} TCPHandler: Code: 43. DB::Exception: First argument for function arrayCount must be a function. Actual Array(Nothing): In scope SELECT arrayCount([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:04.336326 [ 2707 ] {32264227-48a8-4fce-aa36-69cbbac2fd61} executeQuery: Code: 43. DB::Exception: First argument for function arrayCumSum must be a function. Actual Array(Nothing): In scope SELECT arrayCumSum([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayCumSum([], [], []);) 2025.09.25 21:10:04.341585 [ 2707 ] {32264227-48a8-4fce-aa36-69cbbac2fd61} TCPHandler: Code: 43. DB::Exception: First argument for function arrayCumSum must be a function. Actual Array(Nothing): In scope SELECT arrayCumSum([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:04.390468 [ 2707 ] {5563642b-1deb-4c18-84bc-b604fb692926} executeQuery: Code: 43. DB::Exception: First argument for function arrayCumSumNonNegative must be a function. Actual Array(Nothing): In scope SELECT arrayCumSumNonNegative([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayCumSumNonNegative([], [], []);) 2025.09.25 21:10:04.394932 [ 2707 ] {5563642b-1deb-4c18-84bc-b604fb692926} TCPHandler: Code: 43. DB::Exception: First argument for function arrayCumSumNonNegative must be a function. Actual Array(Nothing): In scope SELECT arrayCumSumNonNegative([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:04.493441 [ 2707 ] {e48a7f60-ace7-485a-bbb0-c55480ff0cbf} executeQuery: Code: 43. DB::Exception: First argument for function arrayDifference must be a function. Actual Array(Nothing): In scope SELECT arrayDifference([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayDifference([], [], []);) 2025.09.25 21:10:04.498296 [ 2707 ] {e48a7f60-ace7-485a-bbb0-c55480ff0cbf} TCPHandler: Code: 43. DB::Exception: First argument for function arrayDifference must be a function. Actual Array(Nothing): In scope SELECT arrayDifference([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:04.560516 [ 2707 ] {47c8a879-c8d3-4cab-a3e8-e71075820a9b} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayDistinct doesn't match: passed 3, should be 1: In scope SELECT arrayDistinct([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayDistinct([], [], []);) 2025.09.25 21:10:04.576452 [ 2707 ] {47c8a879-c8d3-4cab-a3e8-e71075820a9b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayDistinct doesn't match: passed 3, should be 1: In scope SELECT arrayDistinct([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:04.674415 [ 2707 ] {3d1a58a0-57e3-41d1-af81-198964651741} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayDotProduct doesn't match: passed 3, should be 2: In scope SELECT arrayDotProduct([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayDotProduct([], [], []);) 2025.09.25 21:10:04.678139 [ 2707 ] {3d1a58a0-57e3-41d1-af81-198964651741} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayDotProduct doesn't match: passed 3, should be 2: In scope SELECT arrayDotProduct([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:04.733529 [ 3355 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {df97ab97-a5cf-4ea8-bd10-026470f5fb5c::202509_1_36_7}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:04.743534 [ 2707 ] {48e7a5ff-22f9-4da7-a57e-a8ac9e71f68b} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayElement doesn't match: passed 3, should be 2: In scope SELECT arrayElement([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayElement([], [], []);) 2025.09.25 21:10:04.762499 [ 2707 ] {48e7a5ff-22f9-4da7-a57e-a8ac9e71f68b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayElement doesn't match: passed 3, should be 2: In scope SELECT arrayElement([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:04.797765 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:04.818702 [ 2707 ] {c0e38ece-1226-45fa-bac6-aed6ec327cde} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayEnumerate doesn't match: passed 3, should be 1: In scope SELECT arrayEnumerate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayEnumerate([], [], []);) 2025.09.25 21:10:04.823409 [ 2707 ] {c0e38ece-1226-45fa-bac6-aed6ec327cde} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayEnumerate doesn't match: passed 3, should be 1: In scope SELECT arrayEnumerate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:04.830378 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:05.566243 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:05.597517 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:05.622410 [ 2707 ] {806c52e8-4382-46fa-ba17-187ba2513ffe} executeQuery: Code: 43. DB::Exception: First argument for function arrayExists must be a function. Actual Array(Nothing): In scope SELECT arrayExists([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayExists([], [], []);) 2025.09.25 21:10:05.636689 [ 2707 ] {806c52e8-4382-46fa-ba17-187ba2513ffe} TCPHandler: Code: 43. DB::Exception: First argument for function arrayExists must be a function. Actual Array(Nothing): In scope SELECT arrayExists([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:05.719476 [ 2707 ] {5828566c-4ba0-4543-a4e8-60101d5b2313} executeQuery: Code: 43. DB::Exception: First argument for function arrayFill must be a function. Actual Array(Nothing): In scope SELECT arrayFill([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayFill([], [], []);) 2025.09.25 21:10:05.736182 [ 2707 ] {5828566c-4ba0-4543-a4e8-60101d5b2313} TCPHandler: Code: 43. DB::Exception: First argument for function arrayFill must be a function. Actual Array(Nothing): In scope SELECT arrayFill([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:05.861990 [ 2707 ] {68920e22-79d9-4e3b-a944-96c276c0b750} executeQuery: Code: 43. DB::Exception: First argument for function arrayFilter must be a function. Actual Array(Nothing): In scope SELECT arrayFilter([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayFilter([], [], []);) 2025.09.25 21:10:05.875687 [ 2707 ] {68920e22-79d9-4e3b-a944-96c276c0b750} TCPHandler: Code: 43. DB::Exception: First argument for function arrayFilter must be a function. Actual Array(Nothing): In scope SELECT arrayFilter([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:05.968848 [ 7178 ] {1aea6fde-45dd-4065-8c07-a4618155b9de} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=1): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:40718) (comment: 00044_any_left_join_string.sql) (in query: SELECT domain, hits, visits FROM ( SELECT domain(URL) AS domain, count() AS hits FROM test.hits GROUP BY domain ) ANY LEFT JOIN ( SELECT domain(StartURL) AS domain, sum(Sign) AS visits FROM test.visits GROUP BY domain ) USING domain ORDER BY hits DESC LIMIT 10 SETTINGS joined_subquery_requires_alias = 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c5f2 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:144: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002def1bcc 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 22. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 23. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:10:05.999686 [ 2707 ] {4ed5de73-477e-42f2-a5d7-a3b63d5fb9cb} executeQuery: Code: 43. DB::Exception: First argument for function arrayFirst must be a function. Actual Array(Nothing): In scope SELECT arrayFirst([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayFirst([], [], []);) 2025.09.25 21:10:06.006706 [ 7178 ] {1aea6fde-45dd-4065-8c07-a4618155b9de} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=1): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c5f2 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:144: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002def1bcc 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 22. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 23. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:10:06.006593 [ 2707 ] {4ed5de73-477e-42f2-a5d7-a3b63d5fb9cb} TCPHandler: Code: 43. DB::Exception: First argument for function arrayFirst must be a function. Actual Array(Nothing): In scope SELECT arrayFirst([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:06.082301 [ 2707 ] {8d925405-1d08-40a7-a4c0-30811f32f5a7} executeQuery: Code: 43. DB::Exception: First argument for function arrayFirstIndex must be a function. Actual Array(Nothing): In scope SELECT arrayFirstIndex([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayFirstIndex([], [], []);) 2025.09.25 21:10:06.088412 [ 2707 ] {8d925405-1d08-40a7-a4c0-30811f32f5a7} TCPHandler: Code: 43. DB::Exception: First argument for function arrayFirstIndex must be a function. Actual Array(Nothing): In scope SELECT arrayFirstIndex([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:06.219025 [ 2707 ] {55d5a45d-ca10-47a0-8de8-88da3e5a71af} executeQuery: Code: 43. DB::Exception: First argument for function arrayFirstOrNull must be a function. Actual Array(Nothing): In scope SELECT arrayFirstOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayFirstOrNull([], [], []);) 2025.09.25 21:10:06.277173 [ 2707 ] {55d5a45d-ca10-47a0-8de8-88da3e5a71af} TCPHandler: Code: 43. DB::Exception: First argument for function arrayFirstOrNull must be a function. Actual Array(Nothing): In scope SELECT arrayFirstOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:06.369581 [ 2707 ] {f1c07eec-1450-4c31-ab56-741720a19da5} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayFlatten doesn't match: passed 3, should be 1: In scope SELECT arrayFlatten([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayFlatten([], [], []);) 2025.09.25 21:10:06.381162 [ 2707 ] {f1c07eec-1450-4c31-ab56-741720a19da5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayFlatten doesn't match: passed 3, should be 1: In scope SELECT arrayFlatten([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:06.514367 [ 2707 ] {459eaaa7-c597-4861-ac36-f13aaec387fc} executeQuery: Code: 43. DB::Exception: First argument for function arrayFold must be a function: In scope SELECT arrayFold([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayFold([], [], []);) 2025.09.25 21:10:06.529437 [ 2707 ] {459eaaa7-c597-4861-ac36-f13aaec387fc} TCPHandler: Code: 43. DB::Exception: First argument for function arrayFold must be a function: In scope SELECT arrayFold([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:06.602772 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:06.634857 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:06.762000 [ 2707 ] {1c3c4801-40b1-477b-8237-fbac15a6d654} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayJaccardIndex doesn't match: passed 3, should be 2: In scope SELECT arrayJaccardIndex([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayJaccardIndex([], [], []);) 2025.09.25 21:10:06.767974 [ 2707 ] {1c3c4801-40b1-477b-8237-fbac15a6d654} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayJaccardIndex doesn't match: passed 3, should be 2: In scope SELECT arrayJaccardIndex([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:06.852748 [ 2707 ] {e808808c-75b2-4aaf-a47d-d0e9b6e06bdc} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayJoin doesn't match: passed 3, should be 1: In scope SELECT arrayJoin([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayJoin([], [], []);) 2025.09.25 21:10:06.854653 [ 7178 ] {fc505843-fa11-4062-b712-62ef33496e5a} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55458) (comment: 00005_filtering.sql) (in query: SELECT count() FROM test.hits WHERE AdvEngineID != 0 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:306: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349fbb 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:06.857158 [ 7178 ] {fc505843-fa11-4062-b712-62ef33496e5a} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=11, jobs=11). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:306: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349fbb 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:06.857470 [ 2707 ] {e808808c-75b2-4aaf-a47d-d0e9b6e06bdc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayJoin doesn't match: passed 3, should be 1: In scope SELECT arrayJoin([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:07.009606 [ 2707 ] {520cb691-6e21-458a-8638-38383805584a} executeQuery: Code: 43. DB::Exception: First argument for function arrayLast must be a function. Actual Array(Nothing): In scope SELECT arrayLast([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayLast([], [], []);) 2025.09.25 21:10:07.016890 [ 2707 ] {520cb691-6e21-458a-8638-38383805584a} TCPHandler: Code: 43. DB::Exception: First argument for function arrayLast must be a function. Actual Array(Nothing): In scope SELECT arrayLast([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:07.100267 [ 2707 ] {ccddbcf4-3a40-4ca1-b8ff-da8c8fc33b59} executeQuery: Code: 43. DB::Exception: First argument for function arrayLastIndex must be a function. Actual Array(Nothing): In scope SELECT arrayLastIndex([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayLastIndex([], [], []);) 2025.09.25 21:10:07.106173 [ 2707 ] {ccddbcf4-3a40-4ca1-b8ff-da8c8fc33b59} TCPHandler: Code: 43. DB::Exception: First argument for function arrayLastIndex must be a function. Actual Array(Nothing): In scope SELECT arrayLastIndex([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:07.222579 [ 2707 ] {9b9c5b67-ec5c-4e83-873a-3a6c8a523f93} executeQuery: Code: 43. DB::Exception: First argument for function arrayLastOrNull must be a function. Actual Array(Nothing): In scope SELECT arrayLastOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayLastOrNull([], [], []);) 2025.09.25 21:10:07.260746 [ 2707 ] {9b9c5b67-ec5c-4e83-873a-3a6c8a523f93} TCPHandler: Code: 43. DB::Exception: First argument for function arrayLastOrNull must be a function. Actual Array(Nothing): In scope SELECT arrayLastOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:07.383668 [ 2707 ] {478a6957-910d-4fd7-839b-81a3d05d668e} executeQuery: Code: 43. DB::Exception: First argument for function arrayMap must be a function. Actual Array(Nothing): In scope SELECT arrayMap([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayMap([], [], []);) 2025.09.25 21:10:07.407403 [ 2707 ] {478a6957-910d-4fd7-839b-81a3d05d668e} TCPHandler: Code: 43. DB::Exception: First argument for function arrayMap must be a function. Actual Array(Nothing): In scope SELECT arrayMap([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:07.524276 [ 2707 ] {0a0e16eb-6fd5-4255-9d84-55209f138785} executeQuery: Code: 43. DB::Exception: First argument for function arrayMax must be a function. Actual Array(Nothing): In scope SELECT arrayMax([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayMax([], [], []);) 2025.09.25 21:10:07.538032 [ 2707 ] {0a0e16eb-6fd5-4255-9d84-55209f138785} TCPHandler: Code: 43. DB::Exception: First argument for function arrayMax must be a function. Actual Array(Nothing): In scope SELECT arrayMax([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:07.646398 [ 2707 ] {a38320ab-6947-4d3d-a0a9-6d3b4018e8bf} executeQuery: Code: 43. DB::Exception: First argument for function arrayMin must be a function. Actual Array(Nothing): In scope SELECT arrayMin([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayMin([], [], []);) 2025.09.25 21:10:07.653817 [ 2707 ] {a38320ab-6947-4d3d-a0a9-6d3b4018e8bf} TCPHandler: Code: 43. DB::Exception: First argument for function arrayMin must be a function. Actual Array(Nothing): In scope SELECT arrayMin([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:07.718025 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:07.757676 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:07.762801 [ 2707 ] {1217f2fd-a5ee-48ca-b758-d0822a44d950} executeQuery: Code: 43. DB::Exception: First argument for function arrayPartialReverseSort must be a function. Actual Array(Nothing): In scope SELECT arrayPartialReverseSort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayPartialReverseSort([], [], []);) 2025.09.25 21:10:07.769091 [ 2707 ] {1217f2fd-a5ee-48ca-b758-d0822a44d950} TCPHandler: Code: 43. DB::Exception: First argument for function arrayPartialReverseSort must be a function. Actual Array(Nothing): In scope SELECT arrayPartialReverseSort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:07.887206 [ 2707 ] {55260eda-aad3-4a39-8274-c9911d197c58} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of arguments of function arrayPartialShuffle (must be UInt or Int): In scope SELECT arrayPartialShuffle([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayPartialShuffle([], [], []);) 2025.09.25 21:10:07.901537 [ 2707 ] {55260eda-aad3-4a39-8274-c9911d197c58} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of arguments of function arrayPartialShuffle (must be UInt or Int): In scope SELECT arrayPartialShuffle([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:08.010035 [ 2707 ] {487e52df-cc03-47bd-804c-d2a0a4465a56} executeQuery: Code: 43. DB::Exception: First argument for function arrayPartialSort must be a function. Actual Array(Nothing): In scope SELECT arrayPartialSort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayPartialSort([], [], []);) 2025.09.25 21:10:08.018944 [ 2707 ] {487e52df-cc03-47bd-804c-d2a0a4465a56} TCPHandler: Code: 43. DB::Exception: First argument for function arrayPartialSort must be a function. Actual Array(Nothing): In scope SELECT arrayPartialSort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:08.113291 [ 2707 ] {66d78790-e593-4223-8f97-ba6e727c3c5b} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayPopBack doesn't match: passed 3, should be 1: In scope SELECT arrayPopBack([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayPopBack([], [], []);) 2025.09.25 21:10:08.122551 [ 2707 ] {66d78790-e593-4223-8f97-ba6e727c3c5b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayPopBack doesn't match: passed 3, should be 1: In scope SELECT arrayPopBack([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:08.226217 [ 2707 ] {94ed8795-1b97-4c79-a79c-e55de3b5204a} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayPopFront doesn't match: passed 3, should be 1: In scope SELECT arrayPopFront([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayPopFront([], [], []);) 2025.09.25 21:10:08.235294 [ 2707 ] {94ed8795-1b97-4c79-a79c-e55de3b5204a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayPopFront doesn't match: passed 3, should be 1: In scope SELECT arrayPopFront([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:08.264875 [ 7178 ] {7b77566c-2d6f-4872-94a7-8828e081631f} executeQuery: Code: 48. DB::Exception: Can't execute any of specified algorithms for specified strictness/kind and right storage type. (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55460) (comment: 00074_full_join.sql) (in query: SELECT CounterID, hits, visits FROM ( SELECT (CounterID % 100000) AS CounterID, count() AS hits FROM test.hits GROUP BY CounterID ) ANY FULL OUTER JOIN ( SELECT (CounterID % 100000) AS CounterID, sum(Sign) AS visits FROM test.visits GROUP BY CounterID HAVING visits > 0 ) USING CounterID WHERE hits = 0 OR visits = 0 ORDER BY hits + visits * 10 DESC, CounterID ASC LIMIT 20;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Planner/PlannerJoins.cpp:917: DB::chooseJoinAlgorithm(std::shared_ptr&, std::shared_ptr const&, DB::Block const&, DB::Block const&, std::shared_ptr const&) @ 0x00000000294a6d9f 5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1490: DB::(anonymous namespace)::buildQueryPlanForJoinNode(std::shared_ptr const&, DB::JoinTreeQueryPlan, DB::JoinTreeQueryPlan, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002947642f 6. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::buildJoinTreeQueryPlan(std::shared_ptr const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002945f174 7. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000002942c986 8. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 9. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 13. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:08.274279 [ 7178 ] {7b77566c-2d6f-4872-94a7-8828e081631f} TCPHandler: Code: 48. DB::Exception: Can't execute any of specified algorithms for specified strictness/kind and right storage type. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Planner/PlannerJoins.cpp:917: DB::chooseJoinAlgorithm(std::shared_ptr&, std::shared_ptr const&, DB::Block const&, DB::Block const&, std::shared_ptr const&) @ 0x00000000294a6d9f 5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1490: DB::(anonymous namespace)::buildQueryPlanForJoinNode(std::shared_ptr const&, DB::JoinTreeQueryPlan, DB::JoinTreeQueryPlan, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002947642f 6. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::buildJoinTreeQueryPlan(std::shared_ptr const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002945f174 7. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000002942c986 8. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 9. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 13. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:08.359498 [ 2707 ] {e73872c6-914e-4571-9944-2a8f1424d542} executeQuery: Code: 43. DB::Exception: First argument for function arrayProduct must be a function. Actual Array(Nothing): In scope SELECT arrayProduct([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayProduct([], [], []);) 2025.09.25 21:10:08.381614 [ 2707 ] {e73872c6-914e-4571-9944-2a8f1424d542} TCPHandler: Code: 43. DB::Exception: First argument for function arrayProduct must be a function. Actual Array(Nothing): In scope SELECT arrayProduct([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:08.495254 [ 2707 ] {5a6e0afa-c508-4207-a881-02e53f8b26c2} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayPushBack doesn't match: passed 3, should be 2: In scope SELECT arrayPushBack([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayPushBack([], [], []);) 2025.09.25 21:10:08.520797 [ 2707 ] {5a6e0afa-c508-4207-a881-02e53f8b26c2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayPushBack doesn't match: passed 3, should be 2: In scope SELECT arrayPushBack([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:08.565876 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:08.604807 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:08.654224 [ 2707 ] {322cf427-1b9b-4a67-8f56-482a8f0b5aa6} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayPushFront doesn't match: passed 3, should be 2: In scope SELECT arrayPushFront([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayPushFront([], [], []);) 2025.09.25 21:10:08.673030 [ 2707 ] {322cf427-1b9b-4a67-8f56-482a8f0b5aa6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayPushFront doesn't match: passed 3, should be 2: In scope SELECT arrayPushFront([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:08.780614 [ 2707 ] {60d39ca6-0e94-4380-b72b-12f9f08a25d2} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayRandomSample doesn't match: passed 3, should be 2: In scope SELECT arrayRandomSample([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayRandomSample([], [], []);) 2025.09.25 21:10:08.788537 [ 2707 ] {60d39ca6-0e94-4380-b72b-12f9f08a25d2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayRandomSample doesn't match: passed 3, should be 2: In scope SELECT arrayRandomSample([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:08.899854 [ 2707 ] {a5f6d989-c951-4734-9ff6-2cf9902db6d8} executeQuery: Code: 43. DB::Exception: First argument for function arrayReduce must be constant string: name of aggregate function.: In scope SELECT arrayReduce([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayReduce([], [], []);) 2025.09.25 21:10:08.908298 [ 2707 ] {a5f6d989-c951-4734-9ff6-2cf9902db6d8} TCPHandler: Code: 43. DB::Exception: First argument for function arrayReduce must be constant string: name of aggregate function.: In scope SELECT arrayReduce([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:09.015778 [ 2707 ] {5e771c3a-7154-4804-b7e3-35185285a214} executeQuery: Code: 43. DB::Exception: First argument for function arrayReduceInRanges must be constant string: name of aggregate function.: In scope SELECT arrayReduceInRanges([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayReduceInRanges([], [], []);) 2025.09.25 21:10:09.028974 [ 2707 ] {5e771c3a-7154-4804-b7e3-35185285a214} TCPHandler: Code: 43. DB::Exception: First argument for function arrayReduceInRanges must be constant string: name of aggregate function.: In scope SELECT arrayReduceInRanges([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:09.070087 [ 3627 ] {7b42d7ef-5a48-4795-92b4-3e774d05a71c} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:52258) (comment: 02950_parallel_replicas_used_count.sql) (in query: DROP DATABASE IF EXISTS test_xym24jv3 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:09.073355 [ 3627 ] {7b42d7ef-5a48-4795-92b4-3e774d05a71c} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:09.174425 [ 2707 ] {b8fcb3a2-7ddd-4649-9b30-7363cd13d377} executeQuery: Code: 43. DB::Exception: Function arrayResize cannot resize Array(Nothing): In scope SELECT arrayResize([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayResize([], [], []);) 2025.09.25 21:10:09.183294 [ 2707 ] {b8fcb3a2-7ddd-4649-9b30-7363cd13d377} TCPHandler: Code: 43. DB::Exception: Function arrayResize cannot resize Array(Nothing): In scope SELECT arrayResize([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:09.307993 [ 2707 ] {e63f753f-841d-4793-a9cb-059a43bfa7d2} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayReverse doesn't match: passed 3, should be 1: In scope SELECT arrayReverse([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayReverse([], [], []);) 2025.09.25 21:10:09.324944 [ 2707 ] {e63f753f-841d-4793-a9cb-059a43bfa7d2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayReverse doesn't match: passed 3, should be 1: In scope SELECT arrayReverse([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:09.343410 [ 7178 ] {1301af38-1878-4837-92cd-88bbbdc661b9} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:52272) (comment: 02950_parallel_replicas_used_count.sql) (in query: DROP DATABASE IF EXISTS test_xym24jv3 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000291f4e6a 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 13. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 14. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 15. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 19. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 20. asan_thread_start(void*) @ 0x000000000aa43059 21. ? @ 0x00007f12cd75eac3 22. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:09.350900 [ 7178 ] {1301af38-1878-4837-92cd-88bbbdc661b9} DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000291f4e6a 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 13. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 14. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 15. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 19. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 20. asan_thread_start(void*) @ 0x000000000aa43059 21. ? @ 0x00007f12cd75eac3 22. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:09.473959 [ 2707 ] {e85aa830-a6dc-4f94-bfbd-e71d6c3c9a4e} executeQuery: Code: 43. DB::Exception: First argument for function arrayReverseFill must be a function. Actual Array(Nothing): In scope SELECT arrayReverseFill([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayReverseFill([], [], []);) 2025.09.25 21:10:09.488433 [ 2707 ] {e85aa830-a6dc-4f94-bfbd-e71d6c3c9a4e} TCPHandler: Code: 43. DB::Exception: First argument for function arrayReverseFill must be a function. Actual Array(Nothing): In scope SELECT arrayReverseFill([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:09.602519 [ 2707 ] {7ebf7df4-bc69-45b8-a94b-5295f07b7144} executeQuery: Code: 43. DB::Exception: First argument for function arrayReverseSort must be a function. Actual Array(Nothing): In scope SELECT arrayReverseSort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayReverseSort([], [], []);) 2025.09.25 21:10:09.614150 [ 2707 ] {7ebf7df4-bc69-45b8-a94b-5295f07b7144} TCPHandler: Code: 43. DB::Exception: First argument for function arrayReverseSort must be a function. Actual Array(Nothing): In scope SELECT arrayReverseSort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:09.787457 [ 2707 ] {83fcdec1-a80a-45b7-bffe-d5c3bc2fc559} executeQuery: Code: 43. DB::Exception: First argument for function arrayReverseSplit must be a function. Actual Array(Nothing): In scope SELECT arrayReverseSplit([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayReverseSplit([], [], []);) 2025.09.25 21:10:09.868325 [ 2707 ] {83fcdec1-a80a-45b7-bffe-d5c3bc2fc559} TCPHandler: Code: 43. DB::Exception: First argument for function arrayReverseSplit must be a function. Actual Array(Nothing): In scope SELECT arrayReverseSplit([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:09.924428 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:09.960711 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:09.968862 [ 2707 ] {43b5824d-575d-4501-9b49-49f9d8c01849} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayRotateLeft doesn't match: passed 3, should be 2: In scope SELECT arrayRotateLeft([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayRotateLeft([], [], []);) 2025.09.25 21:10:09.986936 [ 2707 ] {43b5824d-575d-4501-9b49-49f9d8c01849} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayRotateLeft doesn't match: passed 3, should be 2: In scope SELECT arrayRotateLeft([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:10.084052 [ 3619 ] {edc648cf-0b49-4e19-ab3c-41af2b2bee63} DatabaseReplicated (test_5): It's not recommended to explicitly specify zookeeper_path and replica_name in ReplicatedMergeTree arguments 2025.09.25 21:10:10.093378 [ 3619 ] {edc648cf-0b49-4e19-ab3c-41af2b2bee63} executeQuery: Code: 80. DB::Exception: Explicit zookeeper_path and replica_name are specified in ReplicatedMergeTree arguments. If you really want to specify it explicitly, then you should use some macros to distinguish different shards and replicas. (INCORRECT_QUERY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55480) (comment: 01378_alter_rename_with_ttl_zookeeper.sql) (in query: CREATE TABLE table_rename_with_ttl (`date1` Date, `value1` String) ENGINE = ReplicatedMergeTree('/clickhouse/{database}/test/table_rename_with_ttl_01378', '1') ORDER BY tuple() SETTINGS index_granularity = 48276, min_bytes_for_wide_part = 375386336, ratio_of_defaults_for_sparse_serialization = 0., replace_long_file_name_to_hash = true, max_file_name_length = 86, min_bytes_for_full_part_storage = 189328666, compact_parts_max_bytes_to_buffer = 2410197, compact_parts_max_granules_to_buffer = 72, compact_parts_merge_max_bytes_to_prefetch_part = 18230755, merge_max_block_size = 9790, old_parts_lifetime = 28., prefer_fetch_merged_part_size_threshold = 10737418240, vertical_merge_algorithm_min_rows_to_activate = 1, vertical_merge_algorithm_min_columns_to_activate = 1, min_merge_bytes_to_use_direct_io = 1, index_granularity_bytes = 12688233, concurrent_part_removal_threshold = 73, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = true, marks_compress_block_size = 21636, primary_key_compress_block_size = 55684), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:878: DB::DatabaseReplicated::checkTableEngine(DB::ASTCreateQuery const&, DB::ASTStorage&, std::shared_ptr) const @ 0x0000000027d5afc7 5. ./build_docker/./src/Databases/DatabaseReplicated.cpp:957: DB::DatabaseReplicated::checkQueryValid(std::shared_ptr const&, std::shared_ptr) const @ 0x0000000027d5e336 6. ./build_docker/./src/Databases/DatabaseReplicated.cpp:999: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5f0de 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:10.106091 [ 2707 ] {06913dec-3ab8-48bb-bc0a-352cbe60f189} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayRotateRight doesn't match: passed 3, should be 2: In scope SELECT arrayRotateRight([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayRotateRight([], [], []);) 2025.09.25 21:10:10.122390 [ 3619 ] {edc648cf-0b49-4e19-ab3c-41af2b2bee63} TCPHandler: Code: 80. DB::Exception: Explicit zookeeper_path and replica_name are specified in ReplicatedMergeTree arguments. If you really want to specify it explicitly, then you should use some macros to distinguish different shards and replicas. (INCORRECT_QUERY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:878: DB::DatabaseReplicated::checkTableEngine(DB::ASTCreateQuery const&, DB::ASTStorage&, std::shared_ptr) const @ 0x0000000027d5afc7 5. ./build_docker/./src/Databases/DatabaseReplicated.cpp:957: DB::DatabaseReplicated::checkQueryValid(std::shared_ptr const&, std::shared_ptr) const @ 0x0000000027d5e336 6. ./build_docker/./src/Databases/DatabaseReplicated.cpp:999: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5f0de 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:10.135229 [ 2707 ] {06913dec-3ab8-48bb-bc0a-352cbe60f189} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayRotateRight doesn't match: passed 3, should be 2: In scope SELECT arrayRotateRight([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:10.259333 [ 2707 ] {08f52525-6378-4502-b762-e89770c033e2} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function arrayShiftLeft, expected Native Integer: In scope SELECT arrayShiftLeft([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayShiftLeft([], [], []);) 2025.09.25 21:10:10.270428 [ 2707 ] {08f52525-6378-4502-b762-e89770c033e2} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function arrayShiftLeft, expected Native Integer: In scope SELECT arrayShiftLeft([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:10.462904 [ 2707 ] {97aab9b8-6b48-4a6e-94e8-ced67142cb0b} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function arrayShiftRight, expected Native Integer: In scope SELECT arrayShiftRight([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayShiftRight([], [], []);) 2025.09.25 21:10:10.468238 [ 2707 ] {97aab9b8-6b48-4a6e-94e8-ced67142cb0b} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function arrayShiftRight, expected Native Integer: In scope SELECT arrayShiftRight([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:10.557668 [ 2707 ] {9ae4fb45-698d-42f4-b0df-b6cb93404d98} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayShingles doesn't match: passed 3, should be 2: In scope SELECT arrayShingles([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayShingles([], [], []);) 2025.09.25 21:10:10.561025 [ 2707 ] {9ae4fb45-698d-42f4-b0df-b6cb93404d98} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayShingles doesn't match: passed 3, should be 2: In scope SELECT arrayShingles([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:10.615174 [ 2707 ] {1c15e16b-c6ca-4d51-b177-78622ee1d549} executeQuery: Code: 42. DB::Exception: Function 'arrayShuffle' needs from 1 to 2 arguments, passed 3.: In scope SELECT arrayShuffle([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayShuffle([], [], []);) 2025.09.25 21:10:10.621418 [ 2707 ] {1c15e16b-c6ca-4d51-b177-78622ee1d549} TCPHandler: Code: 42. DB::Exception: Function 'arrayShuffle' needs from 1 to 2 arguments, passed 3.: In scope SELECT arrayShuffle([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:10.688332 [ 2707 ] {f371f223-cdf7-4592-86b8-4f318e7d9039} executeQuery: Code: 43. DB::Exception: Argument 1 for function arraySlice must be integer but it has type Array(Nothing).: In scope SELECT arraySlice([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arraySlice([], [], []);) 2025.09.25 21:10:10.692201 [ 2707 ] {f371f223-cdf7-4592-86b8-4f318e7d9039} TCPHandler: Code: 43. DB::Exception: Argument 1 for function arraySlice must be integer but it has type Array(Nothing).: In scope SELECT arraySlice([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:10.756496 [ 2707 ] {04a767a1-0800-4690-befd-36fd0db9ee0b} executeQuery: Code: 43. DB::Exception: First argument for function arraySort must be a function. Actual Array(Nothing): In scope SELECT arraySort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arraySort([], [], []);) 2025.09.25 21:10:10.792109 [ 2707 ] {04a767a1-0800-4690-befd-36fd0db9ee0b} TCPHandler: Code: 43. DB::Exception: First argument for function arraySort must be a function. Actual Array(Nothing): In scope SELECT arraySort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:10.885244 [ 2707 ] {c53d38a0-6bbf-46d6-9613-559df4a352ac} executeQuery: Code: 43. DB::Exception: First argument for function arraySplit must be a function. Actual Array(Nothing): In scope SELECT arraySplit([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arraySplit([], [], []);) 2025.09.25 21:10:10.892603 [ 2707 ] {c53d38a0-6bbf-46d6-9613-559df4a352ac} TCPHandler: Code: 43. DB::Exception: First argument for function arraySplit must be a function. Actual Array(Nothing): In scope SELECT arraySplit([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:11.021763 [ 2707 ] {4ab97838-b020-4be1-8401-ea10692c929c} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'arrayStringConcat'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT arrayStringConcat([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayStringConcat([], [], []);) 2025.09.25 21:10:11.029954 [ 2707 ] {4ab97838-b020-4be1-8401-ea10692c929c} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'arrayStringConcat'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT arrayStringConcat([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:11.063344 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:11.096941 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:11.100574 [ 2707 ] {90535b7d-cad8-4d4e-bc01-a5c054194506} executeQuery: Code: 43. DB::Exception: First argument for function arraySum must be a function. Actual Array(Nothing): In scope SELECT arraySum([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arraySum([], [], []);) 2025.09.25 21:10:11.106618 [ 2707 ] {90535b7d-cad8-4d4e-bc01-a5c054194506} TCPHandler: Code: 43. DB::Exception: First argument for function arraySum must be a function. Actual Array(Nothing): In scope SELECT arraySum([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:11.310482 [ 2707 ] {b06d46ee-eb00-4ae5-8e13-93382e785193} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayWithConstant doesn't match: passed 3, should be 2: In scope SELECT arrayWithConstant([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT arrayWithConstant([], [], []);) 2025.09.25 21:10:11.314477 [ 2707 ] {b06d46ee-eb00-4ae5-8e13-93382e785193} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayWithConstant doesn't match: passed 3, should be 2: In scope SELECT arrayWithConstant([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:11.412918 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:11.523832 [ 2707 ] {a55df193-6548-486e-8419-dd59c6b1483f} executeQuery: Code: 42. DB::Exception: Aggregate function groupArray requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT array_agg([], [], []);) 2025.09.25 21:10:11.534059 [ 2707 ] {a55df193-6548-486e-8419-dd59c6b1483f} TCPHandler: Code: 42. DB::Exception: Aggregate function groupArray requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:11.538555 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:11.674188 [ 2707 ] {2274ed61-3c3c-4a02-bb43-dcdbf627c93f} executeQuery: Code: 42. DB::Exception: Aggregate function groupArray requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT array_concat_agg([], [], []);) 2025.09.25 21:10:11.680324 [ 2707 ] {2274ed61-3c3c-4a02-bb43-dcdbf627c93f} TCPHandler: Code: 42. DB::Exception: Aggregate function groupArray requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:11.785384 [ 2707 ] {4f90a357-6068-4afe-a601-8fc134516ac2} executeQuery: Code: 42. DB::Exception: Number of arguments for function ascii doesn't match: passed 3, should be 1: In scope SELECT ascii([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ascii([], [], []);) 2025.09.25 21:10:11.790802 [ 2707 ] {4f90a357-6068-4afe-a601-8fc134516ac2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function ascii doesn't match: passed 3, should be 1: In scope SELECT ascii([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:11.847463 [ 2707 ] {5323c890-35d7-4d9c-8285-0a41f64fec58} executeQuery: Code: 42. DB::Exception: Number of arguments for function asin doesn't match: passed 3, should be 1: In scope SELECT asin([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT asin([], [], []);) 2025.09.25 21:10:11.853881 [ 2707 ] {5323c890-35d7-4d9c-8285-0a41f64fec58} TCPHandler: Code: 42. DB::Exception: Number of arguments for function asin doesn't match: passed 3, should be 1: In scope SELECT asin([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:12.030961 [ 2707 ] {8c4750e3-5e4e-4a63-934c-28afbab1c16f} executeQuery: Code: 42. DB::Exception: Number of arguments for function asinh doesn't match: passed 3, should be 1: In scope SELECT asinh([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT asinh([], [], []);) 2025.09.25 21:10:12.038501 [ 2707 ] {8c4750e3-5e4e-4a63-934c-28afbab1c16f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function asinh doesn't match: passed 3, should be 1: In scope SELECT asinh([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:12.097989 [ 2707 ] {950c9d3f-78cd-450a-bce6-0b6264d820b1} executeQuery: Code: 42. DB::Exception: Number of arguments for function assumeNotNull doesn't match: passed 3, should be 1: In scope SELECT assumeNotNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT assumeNotNull([], [], []);) 2025.09.25 21:10:12.185052 [ 2707 ] {950c9d3f-78cd-450a-bce6-0b6264d820b1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function assumeNotNull doesn't match: passed 3, should be 1: In scope SELECT assumeNotNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:12.332380 [ 2707 ] {ab8a8652-a523-4788-9b76-45497965abc5} executeQuery: Code: 42. DB::Exception: Number of arguments for function atan doesn't match: passed 3, should be 1: In scope SELECT atan([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT atan([], [], []);) 2025.09.25 21:10:12.340601 [ 2707 ] {ab8a8652-a523-4788-9b76-45497965abc5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function atan doesn't match: passed 3, should be 1: In scope SELECT atan([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:12.406227 [ 2707 ] {dd459428-8b10-4c63-88e8-ed0a66bc4f9c} executeQuery: Code: 42. DB::Exception: Number of arguments for function atan2 doesn't match: passed 3, should be 2: In scope SELECT atan2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT atan2([], [], []);) 2025.09.25 21:10:12.409254 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:12.412376 [ 2707 ] {dd459428-8b10-4c63-88e8-ed0a66bc4f9c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function atan2 doesn't match: passed 3, should be 2: In scope SELECT atan2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:12.463074 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:12.630389 [ 2707 ] {ea2c0a7f-89e6-4951-a075-75d52097016f} executeQuery: Code: 42. DB::Exception: Number of arguments for function atanh doesn't match: passed 3, should be 1: In scope SELECT atanh([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT atanh([], [], []);) 2025.09.25 21:10:12.638369 [ 2707 ] {ea2c0a7f-89e6-4951-a075-75d52097016f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function atanh doesn't match: passed 3, should be 1: In scope SELECT atanh([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:12.713792 [ 2707 ] {5161a8f5-c976-411d-b19c-354441c040d6} executeQuery: Code: 42. DB::Exception: Aggregate function avg requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT avg([], [], []);) 2025.09.25 21:10:12.718129 [ 2707 ] {5161a8f5-c976-411d-b19c-354441c040d6} TCPHandler: Code: 42. DB::Exception: Aggregate function avg requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:12.812434 [ 2707 ] {3933b942-e7b3-401c-ab3e-a3f6e3c7d620} executeQuery: Code: 42. DB::Exception: Aggregate function avgWeighted requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT avgWeighted([], [], []);) 2025.09.25 21:10:12.815455 [ 2707 ] {3933b942-e7b3-401c-ab3e-a3f6e3c7d620} TCPHandler: Code: 42. DB::Exception: Aggregate function avgWeighted requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:12.842593 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:12.872066 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:12.908390 [ 2707 ] {b75fe8c1-277e-4447-9ba4-a451b2ab2ccd} executeQuery: Code: 43. DB::Exception: All arguments for function bar must be numeric.: In scope SELECT bar([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bar([], [], []);) 2025.09.25 21:10:12.912720 [ 2707 ] {b75fe8c1-277e-4447-9ba4-a451b2ab2ccd} TCPHandler: Code: 43. DB::Exception: All arguments for function bar must be numeric.: In scope SELECT bar([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:12.987838 [ 2707 ] {d2c51e3f-b03e-4f0f-ab15-dd0fa27db710} executeQuery: Code: 42. DB::Exception: Number of arguments for function base58Decode doesn't match: passed 3, should be 1: In scope SELECT base58Decode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT base58Decode([], [], []);) 2025.09.25 21:10:12.998156 [ 2707 ] {d2c51e3f-b03e-4f0f-ab15-dd0fa27db710} TCPHandler: Code: 42. DB::Exception: Number of arguments for function base58Decode doesn't match: passed 3, should be 1: In scope SELECT base58Decode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:13.107439 [ 2707 ] {98d0f739-6b4e-4a4d-8a79-91e0c27b33de} executeQuery: Code: 42. DB::Exception: Number of arguments for function base58Encode doesn't match: passed 3, should be 1: In scope SELECT base58Encode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT base58Encode([], [], []);) 2025.09.25 21:10:13.113820 [ 2707 ] {98d0f739-6b4e-4a4d-8a79-91e0c27b33de} TCPHandler: Code: 42. DB::Exception: Number of arguments for function base58Encode doesn't match: passed 3, should be 1: In scope SELECT base58Encode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:13.173601 [ 2707 ] {c45b0453-d24c-4d2e-b807-78809fcb133c} executeQuery: Code: 42. DB::Exception: Number of arguments for function base64Decode doesn't match: passed 3, should be 1: In scope SELECT base64Decode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT base64Decode([], [], []);) 2025.09.25 21:10:13.178698 [ 2707 ] {c45b0453-d24c-4d2e-b807-78809fcb133c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function base64Decode doesn't match: passed 3, should be 1: In scope SELECT base64Decode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:13.203379 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:13.217563 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:13.231452 [ 2707 ] {a86dd76e-6aba-421a-9339-b9c57a0c0114} executeQuery: Code: 42. DB::Exception: Number of arguments for function base64Encode doesn't match: passed 3, should be 1: In scope SELECT base64Encode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT base64Encode([], [], []);) 2025.09.25 21:10:13.235397 [ 2707 ] {a86dd76e-6aba-421a-9339-b9c57a0c0114} TCPHandler: Code: 42. DB::Exception: Number of arguments for function base64Encode doesn't match: passed 3, should be 1: In scope SELECT base64Encode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:13.307771 [ 2707 ] {aae6a2d5-1164-430a-975c-9ea7cf7b6c41} executeQuery: Code: 42. DB::Exception: Number of arguments for function base64URLDecode doesn't match: passed 3, should be 1: In scope SELECT base64URLDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT base64URLDecode([], [], []);) 2025.09.25 21:10:13.311377 [ 2707 ] {aae6a2d5-1164-430a-975c-9ea7cf7b6c41} TCPHandler: Code: 42. DB::Exception: Number of arguments for function base64URLDecode doesn't match: passed 3, should be 1: In scope SELECT base64URLDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:13.505549 [ 2707 ] {fe64a496-5e3f-4784-9b03-74320044799c} executeQuery: Code: 42. DB::Exception: Number of arguments for function base64URLEncode doesn't match: passed 3, should be 1: In scope SELECT base64URLEncode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT base64URLEncode([], [], []);) 2025.09.25 21:10:13.515817 [ 2707 ] {fe64a496-5e3f-4784-9b03-74320044799c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function base64URLEncode doesn't match: passed 3, should be 1: In scope SELECT base64URLEncode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:13.684968 [ 2707 ] {84b9fabb-22f5-4a38-a8c7-e060c92df8c3} executeQuery: Code: 42. DB::Exception: Number of arguments for function basename doesn't match: passed 3, should be 1: In scope SELECT basename([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT basename([], [], []);) 2025.09.25 21:10:13.690158 [ 2707 ] {84b9fabb-22f5-4a38-a8c7-e060c92df8c3} TCPHandler: Code: 42. DB::Exception: Number of arguments for function basename doesn't match: passed 3, should be 1: In scope SELECT basename([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:13.778257 [ 2707 ] {7ce7cdb6-cd06-4ddb-a08b-7327d892f17b} executeQuery: Code: 42. DB::Exception: Number of arguments for function bin doesn't match: passed 3, should be 1: In scope SELECT bin([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bin([], [], []);) 2025.09.25 21:10:13.785017 [ 2707 ] {7ce7cdb6-cd06-4ddb-a08b-7327d892f17b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bin doesn't match: passed 3, should be 1: In scope SELECT bin([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:13.916068 [ 2707 ] {fc5065c3-c0f1-43f4-a920-d9fc0aea45bc} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitAnd doesn't match: passed 3, should be 2: In scope SELECT bitAnd([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitAnd([], [], []);) 2025.09.25 21:10:13.933087 [ 2707 ] {fc5065c3-c0f1-43f4-a920-d9fc0aea45bc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitAnd doesn't match: passed 3, should be 2: In scope SELECT bitAnd([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:14.014444 [ 2707 ] {82199d75-ea20-48d5-a5b6-bd91643e0b01} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitCount doesn't match: passed 3, should be 1: In scope SELECT bitCount([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitCount([], [], []);) 2025.09.25 21:10:14.024844 [ 2707 ] {82199d75-ea20-48d5-a5b6-bd91643e0b01} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitCount doesn't match: passed 3, should be 1: In scope SELECT bitCount([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:14.053525 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:14.108698 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:14.151997 [ 2707 ] {96bd3694-f303-49f8-b152-bbce92a216f9} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitHammingDistance doesn't match: passed 3, should be 2: In scope SELECT bitHammingDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitHammingDistance([], [], []);) 2025.09.25 21:10:14.159375 [ 2707 ] {96bd3694-f303-49f8-b152-bbce92a216f9} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitHammingDistance doesn't match: passed 3, should be 2: In scope SELECT bitHammingDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:14.257592 [ 7168 ] {d24ebf7c-338e-4e6c-87dd-0b82473963cb} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=3): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55486) (comment: 02306_part_types_profile_events) (in query: SELECT count(), sum(ProfileEvents['InsertedWideParts']), sum(ProfileEvents['InsertedCompactParts']) FROM system.query_log WHERE current_database = currentDatabase() AND log_comment = '02306_part_types_profile_events' AND query ILIKE 'INSERT INTO%' AND type = 'QueryFinish';), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:144: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002def1bcc 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 17. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 18. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 19. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 22. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 23. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 25. asan_thread_start(void*) @ 0x000000000aa43059 26. ? @ 0x00007f12cd75eac3 27. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:14.269379 [ 2707 ] {53ed2d63-bf0b-45cb-8ff4-80abecd624b4} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitNot doesn't match: passed 3, should be 1: In scope SELECT bitNot([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitNot([], [], []);) 2025.09.25 21:10:14.278981 [ 2707 ] {53ed2d63-bf0b-45cb-8ff4-80abecd624b4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitNot doesn't match: passed 3, should be 1: In scope SELECT bitNot([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:14.303331 [ 7168 ] {d24ebf7c-338e-4e6c-87dd-0b82473963cb} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=3): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:144: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002def1bcc 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 17. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 18. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 19. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 22. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 23. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 25. asan_thread_start(void*) @ 0x000000000aa43059 26. ? @ 0x00007f12cd75eac3 27. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:14.405527 [ 2707 ] {b90d69e3-8be9-4b36-96b4-6619b0cddeea} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitOr doesn't match: passed 3, should be 2: In scope SELECT bitOr([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitOr([], [], []);) 2025.09.25 21:10:14.420314 [ 2707 ] {b90d69e3-8be9-4b36-96b4-6619b0cddeea} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitOr doesn't match: passed 3, should be 2: In scope SELECT bitOr([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:14.503253 [ 2707 ] {06ec78ca-a515-4d98-b2cf-299ec2471347} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitPositionsToArray doesn't match: passed 3, should be 1: In scope SELECT bitPositionsToArray([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitPositionsToArray([], [], []);) 2025.09.25 21:10:14.516810 [ 2707 ] {06ec78ca-a515-4d98-b2cf-299ec2471347} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitPositionsToArray doesn't match: passed 3, should be 1: In scope SELECT bitPositionsToArray([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:14.546264 [ 3619 ] {ee765fed-89ce-4142-8018-5f36c5adb70e} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55524) (comment: 00021_2_select_with_in.sql) (in query: select sum(Sign) from test.visits where CounterID in (942285, 577322);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:14.558488 [ 3619 ] {ee765fed-89ce-4142-8018-5f36c5adb70e} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:14.652787 [ 2707 ] {4f1796d6-4354-4ec5-8e22-6dccfec8ff1a} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitRotateLeft doesn't match: passed 3, should be 2: In scope SELECT bitRotateLeft([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitRotateLeft([], [], []);) 2025.09.25 21:10:14.663366 [ 2707 ] {4f1796d6-4354-4ec5-8e22-6dccfec8ff1a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitRotateLeft doesn't match: passed 3, should be 2: In scope SELECT bitRotateLeft([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:14.791115 [ 2707 ] {7865e208-dcd7-473b-adf0-1a1694ed3511} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitRotateRight doesn't match: passed 3, should be 2: In scope SELECT bitRotateRight([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitRotateRight([], [], []);) 2025.09.25 21:10:14.796062 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:14.822072 [ 2707 ] {7865e208-dcd7-473b-adf0-1a1694ed3511} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitRotateRight doesn't match: passed 3, should be 2: In scope SELECT bitRotateRight([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:14.871569 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:14.942259 [ 2707 ] {47d6a177-522c-4c24-8ba8-da25f1424c86} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitShiftLeft doesn't match: passed 3, should be 2: In scope SELECT bitShiftLeft([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitShiftLeft([], [], []);) 2025.09.25 21:10:14.953616 [ 2707 ] {47d6a177-522c-4c24-8ba8-da25f1424c86} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitShiftLeft doesn't match: passed 3, should be 2: In scope SELECT bitShiftLeft([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:15.152528 [ 2707 ] {c8859f95-9c9f-4e70-b333-74d9a69c7496} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitShiftRight doesn't match: passed 3, should be 2: In scope SELECT bitShiftRight([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitShiftRight([], [], []);) 2025.09.25 21:10:15.155086 [ 2707 ] {c8859f95-9c9f-4e70-b333-74d9a69c7496} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitShiftRight doesn't match: passed 3, should be 2: In scope SELECT bitShiftRight([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:15.206486 [ 2707 ] {b3526cea-7af4-4635-87a1-58daaf53b0af} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 's' to function 'bitSlice'. Expected: String, got: Array(Nothing): In scope SELECT bitSlice([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitSlice([], [], []);) 2025.09.25 21:10:15.211112 [ 2707 ] {b3526cea-7af4-4635-87a1-58daaf53b0af} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 's' to function 'bitSlice'. Expected: String, got: Array(Nothing): In scope SELECT bitSlice([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:15.224004 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:15.258324 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:15.295843 [ 2707 ] {bf90a1f7-1133-4146-a90b-5da1991eb951} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitTest doesn't match: passed 3, should be 2: In scope SELECT bitTest([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitTest([], [], []);) 2025.09.25 21:10:15.301886 [ 2707 ] {bf90a1f7-1133-4146-a90b-5da1991eb951} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitTest doesn't match: passed 3, should be 2: In scope SELECT bitTest([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:15.434008 [ 2707 ] {2fa3c87e-64be-4430-a45c-6b52b90aaa3c} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function bitTestAll: In scope SELECT bitTestAll([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitTestAll([], [], []);) 2025.09.25 21:10:15.457938 [ 2707 ] {2fa3c87e-64be-4430-a45c-6b52b90aaa3c} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function bitTestAll: In scope SELECT bitTestAll([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:15.582702 [ 2707 ] {45841414-a61c-4857-bfdb-bc3bb617990d} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function bitTestAny: In scope SELECT bitTestAny([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitTestAny([], [], []);) 2025.09.25 21:10:15.588910 [ 2707 ] {45841414-a61c-4857-bfdb-bc3bb617990d} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function bitTestAny: In scope SELECT bitTestAny([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:15.690648 [ 2707 ] {376bb77c-a1a3-4c40-89c0-c5524acc9d80} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitXor doesn't match: passed 3, should be 2: In scope SELECT bitXor([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitXor([], [], []);) 2025.09.25 21:10:15.697257 [ 2707 ] {376bb77c-a1a3-4c40-89c0-c5524acc9d80} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitXor doesn't match: passed 3, should be 2: In scope SELECT bitXor([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:15.917326 [ 2707 ] {bb5212a8-5a68-40c4-b4b7-cb1ae800f2f6} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapAnd doesn't match: passed 3, should be 2: In scope SELECT bitmapAnd([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapAnd([], [], []);) 2025.09.25 21:10:15.935975 [ 2707 ] {bb5212a8-5a68-40c4-b4b7-cb1ae800f2f6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapAnd doesn't match: passed 3, should be 2: In scope SELECT bitmapAnd([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:16.053917 [ 2707 ] {b0bb523c-d561-4252-89b2-d5083f8943a9} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapAndCardinality doesn't match: passed 3, should be 2: In scope SELECT bitmapAndCardinality([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapAndCardinality([], [], []);) 2025.09.25 21:10:16.064961 [ 2707 ] {b0bb523c-d561-4252-89b2-d5083f8943a9} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapAndCardinality doesn't match: passed 3, should be 2: In scope SELECT bitmapAndCardinality([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:16.160590 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:16.228551 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:16.246074 [ 2707 ] {00f72682-b573-4693-99e5-d755c1dbf3e7} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapAndnot doesn't match: passed 3, should be 2: In scope SELECT bitmapAndnot([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapAndnot([], [], []);) 2025.09.25 21:10:16.254925 [ 2707 ] {00f72682-b573-4693-99e5-d755c1dbf3e7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapAndnot doesn't match: passed 3, should be 2: In scope SELECT bitmapAndnot([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:16.327553 [ 2707 ] {a0dd8063-5cb1-4387-a6ba-c1aac43c5fb6} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapAndnotCardinality doesn't match: passed 3, should be 2: In scope SELECT bitmapAndnotCardinality([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapAndnotCardinality([], [], []);) 2025.09.25 21:10:16.335876 [ 2707 ] {a0dd8063-5cb1-4387-a6ba-c1aac43c5fb6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapAndnotCardinality doesn't match: passed 3, should be 2: In scope SELECT bitmapAndnotCardinality([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:16.386220 [ 2707 ] {495e7528-c2bc-4f8e-a4b4-a6713050f013} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapBuild doesn't match: passed 3, should be 1: In scope SELECT bitmapBuild([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapBuild([], [], []);) 2025.09.25 21:10:16.399164 [ 2707 ] {495e7528-c2bc-4f8e-a4b4-a6713050f013} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapBuild doesn't match: passed 3, should be 1: In scope SELECT bitmapBuild([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:16.510210 [ 2707 ] {7ef3e4e4-7c34-46fa-a9cc-7210866760b9} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapCardinality doesn't match: passed 3, should be 1: In scope SELECT bitmapCardinality([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapCardinality([], [], []);) 2025.09.25 21:10:16.514826 [ 2707 ] {7ef3e4e4-7c34-46fa-a9cc-7210866760b9} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapCardinality doesn't match: passed 3, should be 1: In scope SELECT bitmapCardinality([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:16.595046 [ 2707 ] {c5e943d9-6091-4a62-83b8-b17c1daa9203} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapContains doesn't match: passed 3, should be 2: In scope SELECT bitmapContains([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapContains([], [], []);) 2025.09.25 21:10:16.606478 [ 2707 ] {c5e943d9-6091-4a62-83b8-b17c1daa9203} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapContains doesn't match: passed 3, should be 2: In scope SELECT bitmapContains([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:16.680933 [ 2707 ] {1a16a390-203c-4599-9c9c-ec7750a71e22} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapHasAll doesn't match: passed 3, should be 2: In scope SELECT bitmapHasAll([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapHasAll([], [], []);) 2025.09.25 21:10:16.688271 [ 2707 ] {1a16a390-203c-4599-9c9c-ec7750a71e22} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapHasAll doesn't match: passed 3, should be 2: In scope SELECT bitmapHasAll([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:16.832971 [ 2707 ] {d26c87f1-f4ce-4ec9-b2fc-1c6295c2c11c} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapHasAny doesn't match: passed 3, should be 2: In scope SELECT bitmapHasAny([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapHasAny([], [], []);) 2025.09.25 21:10:16.839763 [ 2707 ] {d26c87f1-f4ce-4ec9-b2fc-1c6295c2c11c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapHasAny doesn't match: passed 3, should be 2: In scope SELECT bitmapHasAny([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:16.938515 [ 2707 ] {7167fb27-16da-40dc-a302-d76229a1f740} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapMax doesn't match: passed 3, should be 1: In scope SELECT bitmapMax([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapMax([], [], []);) 2025.09.25 21:10:16.945005 [ 2707 ] {7167fb27-16da-40dc-a302-d76229a1f740} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapMax doesn't match: passed 3, should be 1: In scope SELECT bitmapMax([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:17.047254 [ 2707 ] {5c65aa3c-3654-4d1e-ade4-4b6c0df595ec} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapMin doesn't match: passed 3, should be 1: In scope SELECT bitmapMin([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapMin([], [], []);) 2025.09.25 21:10:17.054806 [ 2707 ] {5c65aa3c-3654-4d1e-ade4-4b6c0df595ec} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapMin doesn't match: passed 3, should be 1: In scope SELECT bitmapMin([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:17.232551 [ 2707 ] {c82703fe-2391-4727-b2f8-0158391009df} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapOr doesn't match: passed 3, should be 2: In scope SELECT bitmapOr([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapOr([], [], []);) 2025.09.25 21:10:17.241882 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:17.243360 [ 2707 ] {c82703fe-2391-4727-b2f8-0158391009df} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapOr doesn't match: passed 3, should be 2: In scope SELECT bitmapOr([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:17.260217 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:17.350815 [ 2707 ] {349bb5c7-85d1-4cc3-9798-619be8c87661} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapOrCardinality doesn't match: passed 3, should be 2: In scope SELECT bitmapOrCardinality([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapOrCardinality([], [], []);) 2025.09.25 21:10:17.366662 [ 2707 ] {349bb5c7-85d1-4cc3-9798-619be8c87661} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapOrCardinality doesn't match: passed 3, should be 2: In scope SELECT bitmapOrCardinality([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:17.450011 [ 2707 ] {814be8cd-3079-4412-9813-a2d7803d14d0} executeQuery: Code: 43. DB::Exception: First argument for function bitmapSubsetInRange must be a bitmap but it has type Array(Nothing).: In scope SELECT bitmapSubsetInRange([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapSubsetInRange([], [], []);) 2025.09.25 21:10:17.457749 [ 2707 ] {814be8cd-3079-4412-9813-a2d7803d14d0} TCPHandler: Code: 43. DB::Exception: First argument for function bitmapSubsetInRange must be a bitmap but it has type Array(Nothing).: In scope SELECT bitmapSubsetInRange([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:17.470956 [ 3619 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:17.578667 [ 2707 ] {919e0014-be74-44d8-aece-a3deb593ff21} executeQuery: Code: 43. DB::Exception: First argument for function bitmapSubsetLimit must be a bitmap but it has type Array(Nothing).: In scope SELECT bitmapSubsetLimit([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapSubsetLimit([], [], []);) 2025.09.25 21:10:17.598465 [ 2707 ] {919e0014-be74-44d8-aece-a3deb593ff21} TCPHandler: Code: 43. DB::Exception: First argument for function bitmapSubsetLimit must be a bitmap but it has type Array(Nothing).: In scope SELECT bitmapSubsetLimit([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:17.718971 [ 2707 ] {c027b40a-80bb-4866-9805-bc914e6ad422} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapToArray doesn't match: passed 3, should be 1: In scope SELECT bitmapToArray([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapToArray([], [], []);) 2025.09.25 21:10:17.722991 [ 2707 ] {c027b40a-80bb-4866-9805-bc914e6ad422} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapToArray doesn't match: passed 3, should be 1: In scope SELECT bitmapToArray([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:17.788600 [ 2707 ] {d02a7ac3-91c2-4bea-b071-5565e7d802a9} executeQuery: Code: 43. DB::Exception: First argument for function bitmapTransform must be a bitmap but it has type Array(Nothing).: In scope SELECT bitmapTransform([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapTransform([], [], []);) 2025.09.25 21:10:17.793029 [ 2707 ] {d02a7ac3-91c2-4bea-b071-5565e7d802a9} TCPHandler: Code: 43. DB::Exception: First argument for function bitmapTransform must be a bitmap but it has type Array(Nothing).: In scope SELECT bitmapTransform([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:17.812256 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:17.845137 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:17.854366 [ 2707 ] {d4aaac97-26ce-4205-8306-08662a1d76fa} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapXor doesn't match: passed 3, should be 2: In scope SELECT bitmapXor([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapXor([], [], []);) 2025.09.25 21:10:17.860218 [ 2707 ] {d4aaac97-26ce-4205-8306-08662a1d76fa} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapXor doesn't match: passed 3, should be 2: In scope SELECT bitmapXor([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:17.922718 [ 2707 ] {8e5f2388-ab1f-45eb-8814-6b2044c5ba31} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmapXorCardinality doesn't match: passed 3, should be 2: In scope SELECT bitmapXorCardinality([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmapXorCardinality([], [], []);) 2025.09.25 21:10:17.929895 [ 2707 ] {8e5f2388-ab1f-45eb-8814-6b2044c5ba31} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmapXorCardinality doesn't match: passed 3, should be 2: In scope SELECT bitmapXorCardinality([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:17.985034 [ 2707 ] {e1ac2875-bf92-461d-8aee-526d02ab5997} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmaskToArray doesn't match: passed 3, should be 1: In scope SELECT bitmaskToArray([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmaskToArray([], [], []);) 2025.09.25 21:10:17.988164 [ 2707 ] {e1ac2875-bf92-461d-8aee-526d02ab5997} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmaskToArray doesn't match: passed 3, should be 1: In scope SELECT bitmaskToArray([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:18.061808 [ 2707 ] {9639bb07-156b-4142-a46d-c5dd894740c7} executeQuery: Code: 42. DB::Exception: Number of arguments for function bitmaskToList doesn't match: passed 3, should be 1: In scope SELECT bitmaskToList([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT bitmaskToList([], [], []);) 2025.09.25 21:10:18.069853 [ 2707 ] {9639bb07-156b-4142-a46d-c5dd894740c7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function bitmaskToList doesn't match: passed 3, should be 1: In scope SELECT bitmaskToList([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:18.230544 [ 2707 ] {ba2fb90c-255f-4035-93b2-620d94041252} executeQuery: Code: 42. DB::Exception: Number of arguments for function blockNumber doesn't match: passed 3, should be 0: In scope SELECT blockNumber([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT blockNumber([], [], []);) 2025.09.25 21:10:18.234916 [ 2707 ] {ba2fb90c-255f-4035-93b2-620d94041252} TCPHandler: Code: 42. DB::Exception: Number of arguments for function blockNumber doesn't match: passed 3, should be 0: In scope SELECT blockNumber([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:18.385342 [ 2707 ] {148636a0-a217-4cfc-9de7-27caa1a9579e} executeQuery: Code: 42. DB::Exception: Number of arguments for function blockSize doesn't match: passed 3, should be 0: In scope SELECT blockSize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT blockSize([], [], []);) 2025.09.25 21:10:18.390974 [ 2707 ] {148636a0-a217-4cfc-9de7-27caa1a9579e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function blockSize doesn't match: passed 3, should be 0: In scope SELECT blockSize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:18.540920 [ 2707 ] {69804eb9-f131-4e95-82de-dfa60d87ccff} executeQuery: Code: 42. DB::Exception: Aggregate function boundingRatio requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT boundingRatio([], [], []);) 2025.09.25 21:10:18.548891 [ 2707 ] {69804eb9-f131-4e95-82de-dfa60d87ccff} TCPHandler: Code: 42. DB::Exception: Aggregate function boundingRatio requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:18.609016 [ 2707 ] {74c6a938-0cde-49b0-942b-94584950a0eb} executeQuery: Code: 42. DB::Exception: Number of arguments for function buildId doesn't match: passed 3, should be 0: In scope SELECT buildId([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT buildId([], [], []);) 2025.09.25 21:10:18.616714 [ 2707 ] {74c6a938-0cde-49b0-942b-94584950a0eb} TCPHandler: Code: 42. DB::Exception: Number of arguments for function buildId doesn't match: passed 3, should be 0: In scope SELECT buildId([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:18.754708 [ 2707 ] {93f5bbd9-6ede-4262-88fc-3e024aec0216} executeQuery: Code: 42. DB::Exception: Number of arguments for function byteHammingDistance doesn't match: passed 3, should be 2: In scope SELECT byteHammingDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT byteHammingDistance([], [], []);) 2025.09.25 21:10:18.767604 [ 2707 ] {93f5bbd9-6ede-4262-88fc-3e024aec0216} TCPHandler: Code: 42. DB::Exception: Number of arguments for function byteHammingDistance doesn't match: passed 3, should be 2: In scope SELECT byteHammingDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:18.994633 [ 2707 ] {a1895e70-83be-4cdc-a09a-dd6ef7d2a8b8} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function substring, expected String, FixedString or Enum: In scope SELECT byteSlice([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT byteSlice([], [], []);) 2025.09.25 21:10:19.004195 [ 2707 ] {a1895e70-83be-4cdc-a09a-dd6ef7d2a8b8} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function substring, expected String, FixedString or Enum: In scope SELECT byteSlice([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:19.054279 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:19.108228 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:19.113061 [ 2707 ] {4028b867-7859-4864-9e24-ce65bb9516ca} executeQuery: Code: 42. DB::Exception: Number of arguments for function byteSwap doesn't match: passed 3, should be 1: In scope SELECT byteSwap([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT byteSwap([], [], []);) 2025.09.25 21:10:19.121809 [ 2707 ] {4028b867-7859-4864-9e24-ce65bb9516ca} TCPHandler: Code: 42. DB::Exception: Number of arguments for function byteSwap doesn't match: passed 3, should be 1: In scope SELECT byteSwap([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:19.350322 [ 2707 ] {0505ba96-adea-413d-aca5-27339dca6d56} executeQuery: Code: 36. DB::Exception: Unexpected type Array(Nothing) in function 'transform': In scope SELECT caseWithExpr([], [], []). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT caseWithExpr([], [], []);) 2025.09.25 21:10:19.354506 [ 2707 ] {0505ba96-adea-413d-aca5-27339dca6d56} TCPHandler: Code: 36. DB::Exception: Unexpected type Array(Nothing) in function 'transform': In scope SELECT caseWithExpr([], [], []). (BAD_ARGUMENTS) 2025.09.25 21:10:19.427821 [ 2707 ] {caec1ffc-2b9e-48cc-ac14-7be409be0214} executeQuery: Code: 36. DB::Exception: Unexpected type Array(Nothing) in function 'transform': In scope SELECT caseWithExpression([], [], []). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT caseWithExpression([], [], []);) 2025.09.25 21:10:19.437243 [ 2707 ] {caec1ffc-2b9e-48cc-ac14-7be409be0214} TCPHandler: Code: 36. DB::Exception: Unexpected type Array(Nothing) in function 'transform': In scope SELECT caseWithExpression([], [], []). (BAD_ARGUMENTS) 2025.09.25 21:10:19.535541 [ 2707 ] {7845d320-6467-4281-a834-0a4046ed94ec} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument (condition) of function multiIf. Must be UInt8.: In scope SELECT caseWithoutExpr([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT caseWithoutExpr([], [], []);) 2025.09.25 21:10:19.551833 [ 2707 ] {7845d320-6467-4281-a834-0a4046ed94ec} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument (condition) of function multiIf. Must be UInt8.: In scope SELECT caseWithoutExpr([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:19.741644 [ 2707 ] {02c94440-7a19-407a-ab34-0b13e97bd077} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument (condition) of function multiIf. Must be UInt8.: In scope SELECT caseWithoutExpression([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT caseWithoutExpression([], [], []);) 2025.09.25 21:10:19.749426 [ 2707 ] {02c94440-7a19-407a-ab34-0b13e97bd077} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument (condition) of function multiIf. Must be UInt8.: In scope SELECT caseWithoutExpression([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:19.780184 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:19.813194 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:19.826698 [ 2707 ] {d8c902b6-4cd5-452d-8cbd-adad5c01ebda} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function catboostEvaluate, expected a string.: In scope SELECT catboostEvaluate([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT catboostEvaluate([], [], []);) 2025.09.25 21:10:19.856385 [ 2707 ] {d8c902b6-4cd5-452d-8cbd-adad5c01ebda} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function catboostEvaluate, expected a string.: In scope SELECT catboostEvaluate([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:19.937139 [ 2707 ] {693a99b2-cdec-47f5-9846-9ded5199791e} executeQuery: Code: 43. DB::Exception: All the arguments of aggregate function categoricalInformationValue should be UInt8. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT categoricalInformationValue([], [], []);) 2025.09.25 21:10:19.953267 [ 2707 ] {693a99b2-cdec-47f5-9846-9ded5199791e} TCPHandler: Code: 43. DB::Exception: All the arguments of aggregate function categoricalInformationValue should be UInt8. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:20.113170 [ 2707 ] {b653b0ae-f42b-44f8-8837-2c3104466046} executeQuery: Code: 42. DB::Exception: Number of arguments for function cbrt doesn't match: passed 3, should be 1: In scope SELECT cbrt([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cbrt([], [], []);) 2025.09.25 21:10:20.119696 [ 2707 ] {b653b0ae-f42b-44f8-8837-2c3104466046} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cbrt doesn't match: passed 3, should be 1: In scope SELECT cbrt([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:20.210301 [ 2707 ] {e9af7461-e504-4806-84bd-ea8f8edc9c50} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'ceil'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT ceil([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ceil([], [], []);) 2025.09.25 21:10:20.224460 [ 2707 ] {e9af7461-e504-4806-84bd-ea8f8edc9c50} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'ceil'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT ceil([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:20.340892 [ 2707 ] {1ae6d73f-419a-40d2-9855-09a8335a0f5c} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'ceil'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT ceiling([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ceiling([], [], []);) 2025.09.25 21:10:20.343465 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:20.350270 [ 2707 ] {1ae6d73f-419a-40d2-9855-09a8335a0f5c} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'ceil'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT ceiling([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:20.399721 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:20.444052 [ 2707 ] {a2715544-497f-485d-8ebb-2160093e9f34} executeQuery: Code: 42. DB::Exception: Number of arguments for function changeDay doesn't match: passed 3, should be 2: In scope SELECT changeDay([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT changeDay([], [], []);) 2025.09.25 21:10:20.459676 [ 2707 ] {a2715544-497f-485d-8ebb-2160093e9f34} TCPHandler: Code: 42. DB::Exception: Number of arguments for function changeDay doesn't match: passed 3, should be 2: In scope SELECT changeDay([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:20.619918 [ 2707 ] {1e906e28-ecb8-4722-8fbb-39fbba91719f} executeQuery: Code: 42. DB::Exception: Number of arguments for function changeHour doesn't match: passed 3, should be 2: In scope SELECT changeHour([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT changeHour([], [], []);) 2025.09.25 21:10:20.637124 [ 2707 ] {1e906e28-ecb8-4722-8fbb-39fbba91719f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function changeHour doesn't match: passed 3, should be 2: In scope SELECT changeHour([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:20.785583 [ 2707 ] {e98874de-45af-4925-86b4-3bd1b7eec04f} executeQuery: Code: 42. DB::Exception: Number of arguments for function changeMinute doesn't match: passed 3, should be 2: In scope SELECT changeMinute([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT changeMinute([], [], []);) 2025.09.25 21:10:20.808041 [ 2707 ] {e98874de-45af-4925-86b4-3bd1b7eec04f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function changeMinute doesn't match: passed 3, should be 2: In scope SELECT changeMinute([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:20.934411 [ 2707 ] {6342234f-cdbb-4878-869f-b3da8e1f26cd} executeQuery: Code: 42. DB::Exception: Number of arguments for function changeMonth doesn't match: passed 3, should be 2: In scope SELECT changeMonth([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT changeMonth([], [], []);) 2025.09.25 21:10:20.948668 [ 2707 ] {6342234f-cdbb-4878-869f-b3da8e1f26cd} TCPHandler: Code: 42. DB::Exception: Number of arguments for function changeMonth doesn't match: passed 3, should be 2: In scope SELECT changeMonth([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:21.050432 [ 2707 ] {2c72cf3f-a2ae-4dd1-a053-7f417baa07f5} executeQuery: Code: 42. DB::Exception: Number of arguments for function changeSecond doesn't match: passed 3, should be 2: In scope SELECT changeSecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT changeSecond([], [], []);) 2025.09.25 21:10:21.057556 [ 2707 ] {2c72cf3f-a2ae-4dd1-a053-7f417baa07f5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function changeSecond doesn't match: passed 3, should be 2: In scope SELECT changeSecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:21.147468 [ 2707 ] {2d55a21d-ec19-456b-8aad-d66e8eee9a43} executeQuery: Code: 42. DB::Exception: Number of arguments for function changeYear doesn't match: passed 3, should be 2: In scope SELECT changeYear([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT changeYear([], [], []);) 2025.09.25 21:10:21.164241 [ 2707 ] {2d55a21d-ec19-456b-8aad-d66e8eee9a43} TCPHandler: Code: 42. DB::Exception: Number of arguments for function changeYear doesn't match: passed 3, should be 2: In scope SELECT changeYear([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:21.187617 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:21.211204 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:21.259184 [ 2707 ] {341b547b-7e8d-48f7-b8d5-4adfca5d400b} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function char, must be Int, UInt or Float number: In scope SELECT char([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT char([], [], []);) 2025.09.25 21:10:21.278252 [ 2707 ] {341b547b-7e8d-48f7-b8d5-4adfca5d400b} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function char, must be Int, UInt or Float number: In scope SELECT char([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:21.862234 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:21.896914 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:22.030099 [ 2707 ] {7739f8e8-f85d-45e6-9833-5c88a4caaf26} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function concatWithSeparator: In scope SELECT concatWithSeparator([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT concatWithSeparator([], [], []);) 2025.09.25 21:10:22.060470 [ 2707 ] {7739f8e8-f85d-45e6-9833-5c88a4caaf26} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function concatWithSeparator: In scope SELECT concatWithSeparator([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:22.192151 [ 2707 ] {b9821b77-a7dd-4fd2-a4ee-edab4ae3d6d3} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function concatWithSeparatorAssumeInjective: In scope SELECT concatWithSeparatorAssumeInjective([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT concatWithSeparatorAssumeInjective([], [], []);) 2025.09.25 21:10:22.210283 [ 2707 ] {b9821b77-a7dd-4fd2-a4ee-edab4ae3d6d3} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function concatWithSeparatorAssumeInjective: In scope SELECT concatWithSeparatorAssumeInjective([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:22.308576 [ 2707 ] {5db8bb64-98d2-4b04-972a-658f55d67be8} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function concatWithSeparator: In scope SELECT concat_ws([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT concat_ws([], [], []);) 2025.09.25 21:10:22.324663 [ 2707 ] {5db8bb64-98d2-4b04-972a-658f55d67be8} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function concatWithSeparator: In scope SELECT concat_ws([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:22.379712 [ 2707 ] {7579de45-df71-4d29-9c78-28d045530abc} executeQuery: Code: 42. DB::Exception: Number of arguments for function connectionId doesn't match: passed 3, should be 0: In scope SELECT connectionId([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT connectionId([], [], []);) 2025.09.25 21:10:22.385965 [ 2707 ] {7579de45-df71-4d29-9c78-28d045530abc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function connectionId doesn't match: passed 3, should be 0: In scope SELECT connectionId([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:22.553055 [ 2707 ] {304b65c6-cc73-43d2-bb27-b9c90d114b22} executeQuery: Code: 42. DB::Exception: Number of arguments for function connectionId doesn't match: passed 3, should be 0: In scope SELECT connection_id([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT connection_id([], [], []);) 2025.09.25 21:10:22.560392 [ 2707 ] {304b65c6-cc73-43d2-bb27-b9c90d114b22} TCPHandler: Code: 42. DB::Exception: Number of arguments for function connectionId doesn't match: passed 3, should be 0: In scope SELECT connection_id([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:22.726006 [ 2707 ] {b5774fb9-15c9-459e-b56d-072b683bb605} executeQuery: Code: 42. DB::Exception: Aggregate function contingency requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT contingency([], [], []);) 2025.09.25 21:10:22.740892 [ 2707 ] {b5774fb9-15c9-459e-b56d-072b683bb605} TCPHandler: Code: 42. DB::Exception: Aggregate function contingency requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:22.741572 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:22.774740 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:22.823979 [ 2707 ] {6a4e15c8-8d4c-4826-adbe-212d406e00fd} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function convertCharset, must be String: In scope SELECT convertCharset([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT convertCharset([], [], []);) 2025.09.25 21:10:22.829949 [ 2707 ] {6a4e15c8-8d4c-4826-adbe-212d406e00fd} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function convertCharset, must be String: In scope SELECT convertCharset([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:22.943556 [ 2707 ] {dbf9ca15-014c-4730-bedd-1e9f7f8679ff} executeQuery: Code: 42. DB::Exception: Aggregate function corr requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT corr([], [], []);) 2025.09.25 21:10:22.955214 [ 2707 ] {dbf9ca15-014c-4730-bedd-1e9f7f8679ff} TCPHandler: Code: 42. DB::Exception: Aggregate function corr requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:23.046544 [ 2707 ] {a49f52f1-fa81-43a8-82aa-715faeee49cb} executeQuery: Code: 43. DB::Exception: Aggregate function corrMatrix only supports numerical types. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT corrMatrix([], [], []);) 2025.09.25 21:10:23.072352 [ 2707 ] {a49f52f1-fa81-43a8-82aa-715faeee49cb} TCPHandler: Code: 43. DB::Exception: Aggregate function corrMatrix only supports numerical types. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:23.163871 [ 2707 ] {167139af-275a-4971-9129-88da6b20afc1} executeQuery: Code: 42. DB::Exception: Aggregate function corrStable requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT corrStable([], [], []);) 2025.09.25 21:10:23.170268 [ 2707 ] {167139af-275a-4971-9129-88da6b20afc1} TCPHandler: Code: 42. DB::Exception: Aggregate function corrStable requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:23.258774 [ 2707 ] {ad6e5d1b-45ed-485a-8fdd-31f1e54a58b5} executeQuery: Code: 42. DB::Exception: Number of arguments for function cos doesn't match: passed 3, should be 1: In scope SELECT cos([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cos([], [], []);) 2025.09.25 21:10:23.264557 [ 2707 ] {ad6e5d1b-45ed-485a-8fdd-31f1e54a58b5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cos doesn't match: passed 3, should be 1: In scope SELECT cos([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:23.355607 [ 2707 ] {2d285c7d-845c-435e-b9a1-ede6fef597ec} executeQuery: Code: 42. DB::Exception: Number of arguments for function cosh doesn't match: passed 3, should be 1: In scope SELECT cosh([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cosh([], [], []);) 2025.09.25 21:10:23.364033 [ 2707 ] {2d285c7d-845c-435e-b9a1-ede6fef597ec} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cosh doesn't match: passed 3, should be 1: In scope SELECT cosh([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:23.492457 [ 2707 ] {e196f49d-c982-43a2-ade0-7dd9c387528b} executeQuery: Code: 42. DB::Exception: Number of arguments for function cosineDistance doesn't match: passed 3, should be 2: In scope SELECT cosineDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cosineDistance([], [], []);) 2025.09.25 21:10:23.500455 [ 2707 ] {e196f49d-c982-43a2-ade0-7dd9c387528b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cosineDistance doesn't match: passed 3, should be 2: In scope SELECT cosineDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:23.541796 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:23.561001 [ 2707 ] {c6b89b20-eb75-44bf-8891-dafb7d1e1b29} executeQuery: Code: 42. DB::Exception: Aggregate function count requires zero or one argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT count([], [], []);) 2025.09.25 21:10:23.566612 [ 2707 ] {c6b89b20-eb75-44bf-8891-dafb7d1e1b29} TCPHandler: Code: 42. DB::Exception: Aggregate function count requires zero or one argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:23.620686 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:23.631058 [ 2707 ] {3dda6398-5753-4098-b4de-a1ec6226180b} executeQuery: Code: 42. DB::Exception: Number of arguments for function countDigits doesn't match: passed 3, should be 1: In scope SELECT countDigits([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT countDigits([], [], []);) 2025.09.25 21:10:23.638803 [ 2707 ] {3dda6398-5753-4098-b4de-a1ec6226180b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function countDigits doesn't match: passed 3, should be 1: In scope SELECT countDigits([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:23.750197 [ 2707 ] {714d1e48-93e5-4475-a21d-562e19c99da6} executeQuery: Code: 42. DB::Exception: Number of arguments for function countEqual doesn't match: passed 3, should be 2: In scope SELECT countEqual([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT countEqual([], [], []);) 2025.09.25 21:10:23.759600 [ 2707 ] {714d1e48-93e5-4475-a21d-562e19c99da6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function countEqual doesn't match: passed 3, should be 2: In scope SELECT countEqual([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:23.864342 [ 2707 ] {5b539d1c-89a5-472e-9426-94cd86b78fa8} executeQuery: Code: 42. DB::Exception: Number of arguments for function countMatches doesn't match: passed 3, should be 2: In scope SELECT countMatches([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT countMatches([], [], []);) 2025.09.25 21:10:23.868899 [ 2707 ] {5b539d1c-89a5-472e-9426-94cd86b78fa8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function countMatches doesn't match: passed 3, should be 2: In scope SELECT countMatches([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:23.954228 [ 2707 ] {1e16a19f-d614-4238-ae0b-407c1eed7445} executeQuery: Code: 42. DB::Exception: Number of arguments for function countMatchesCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT countMatchesCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT countMatchesCaseInsensitive([], [], []);) 2025.09.25 21:10:23.959420 [ 2707 ] {1e16a19f-d614-4238-ae0b-407c1eed7445} TCPHandler: Code: 42. DB::Exception: Number of arguments for function countMatchesCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT countMatchesCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:24.071667 [ 2707 ] {c8cb5fbb-5ea2-460f-8d44-6525a4145646} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function countSubstrings: In scope SELECT countSubstrings([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT countSubstrings([], [], []);) 2025.09.25 21:10:24.083805 [ 2707 ] {c8cb5fbb-5ea2-460f-8d44-6525a4145646} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function countSubstrings: In scope SELECT countSubstrings([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:24.164592 [ 2707 ] {705322b1-141a-4047-8ede-b4624340b8bf} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function countSubstringsCaseInsensitive: In scope SELECT countSubstringsCaseInsensitive([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT countSubstringsCaseInsensitive([], [], []);) 2025.09.25 21:10:24.169477 [ 2707 ] {705322b1-141a-4047-8ede-b4624340b8bf} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function countSubstringsCaseInsensitive: In scope SELECT countSubstringsCaseInsensitive([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:24.299374 [ 2707 ] {bb993926-61d1-42e9-86ed-e476687c5e69} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function countSubstringsCaseInsensitiveUTF8: In scope SELECT countSubstringsCaseInsensitiveUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT countSubstringsCaseInsensitiveUTF8([], [], []);) 2025.09.25 21:10:24.304278 [ 2707 ] {bb993926-61d1-42e9-86ed-e476687c5e69} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function countSubstringsCaseInsensitiveUTF8: In scope SELECT countSubstringsCaseInsensitiveUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:24.379647 [ 2707 ] {88817622-8c81-40d7-9f36-ac48632ec072} executeQuery: Code: 42. DB::Exception: Aggregate function covarPop requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT covarPop([], [], []);) 2025.09.25 21:10:24.386439 [ 2707 ] {88817622-8c81-40d7-9f36-ac48632ec072} TCPHandler: Code: 42. DB::Exception: Aggregate function covarPop requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:24.519536 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:24.550681 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:24.569030 [ 2707 ] {94da9c63-bff3-4e09-bc5e-fe1f6e2966d8} executeQuery: Code: 43. DB::Exception: Aggregate function covarPopMatrix only supports numerical types. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT covarPopMatrix([], [], []);) 2025.09.25 21:10:24.577790 [ 2707 ] {94da9c63-bff3-4e09-bc5e-fe1f6e2966d8} TCPHandler: Code: 43. DB::Exception: Aggregate function covarPopMatrix only supports numerical types. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:24.751718 [ 2707 ] {8175c173-785c-4508-acac-c999fbf34d05} executeQuery: Code: 42. DB::Exception: Aggregate function covarPopStable requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT covarPopStable([], [], []);) 2025.09.25 21:10:24.756769 [ 2707 ] {8175c173-785c-4508-acac-c999fbf34d05} TCPHandler: Code: 42. DB::Exception: Aggregate function covarPopStable requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:24.906197 [ 2707 ] {c86181d1-14c2-4548-b447-8879aa9dc687} executeQuery: Code: 42. DB::Exception: Aggregate function covarSamp requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT covarSamp([], [], []);) 2025.09.25 21:10:24.911514 [ 2707 ] {c86181d1-14c2-4548-b447-8879aa9dc687} TCPHandler: Code: 42. DB::Exception: Aggregate function covarSamp requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:24.983997 [ 2707 ] {3c656524-e7b7-4c8e-a64b-4f1b5a208497} executeQuery: Code: 43. DB::Exception: Aggregate function covarSampMatrix only supports numerical types. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT covarSampMatrix([], [], []);) 2025.09.25 21:10:24.993204 [ 2707 ] {3c656524-e7b7-4c8e-a64b-4f1b5a208497} TCPHandler: Code: 43. DB::Exception: Aggregate function covarSampMatrix only supports numerical types. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:25.088086 [ 2707 ] {a81d47ba-81d1-4a62-99a7-27f98a1b5687} executeQuery: Code: 42. DB::Exception: Aggregate function covarSampStable requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT covarSampStable([], [], []);) 2025.09.25 21:10:25.092800 [ 2707 ] {a81d47ba-81d1-4a62-99a7-27f98a1b5687} TCPHandler: Code: 42. DB::Exception: Aggregate function covarSampStable requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:25.161870 [ 2707 ] {633e180f-2a12-4512-ba15-ff041f2976fd} executeQuery: Code: 42. DB::Exception: Aggregate function cramersV requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cramersV([], [], []);) 2025.09.25 21:10:25.172751 [ 2707 ] {633e180f-2a12-4512-ba15-ff041f2976fd} TCPHandler: Code: 42. DB::Exception: Aggregate function cramersV requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:25.238862 [ 2707 ] {963c1029-054f-427e-8a51-299456c0db71} executeQuery: Code: 42. DB::Exception: Aggregate function cramersVBiasCorrected requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cramersVBiasCorrected([], [], []);) 2025.09.25 21:10:25.250790 [ 2707 ] {963c1029-054f-427e-8a51-299456c0db71} TCPHandler: Code: 42. DB::Exception: Aggregate function cramersVBiasCorrected requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:25.342934 [ 2707 ] {0ae52053-226f-4cc7-8e3a-104c77e3c493} executeQuery: Code: 42. DB::Exception: Number of arguments for function today doesn't match: passed 3, should be 0: In scope SELECT curdate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT curdate([], [], []);) 2025.09.25 21:10:25.356198 [ 2707 ] {0ae52053-226f-4cc7-8e3a-104c77e3c493} TCPHandler: Code: 42. DB::Exception: Number of arguments for function today doesn't match: passed 3, should be 0: In scope SELECT curdate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:25.412139 [ 2707 ] {034f026a-f8d3-46b9-b9fd-63390dca777a} executeQuery: Code: 42. DB::Exception: Number of arguments for function currentDatabase doesn't match: passed 3, should be 0: In scope SELECT currentDatabase([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT currentDatabase([], [], []);) 2025.09.25 21:10:25.416703 [ 2707 ] {034f026a-f8d3-46b9-b9fd-63390dca777a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentDatabase doesn't match: passed 3, should be 0: In scope SELECT currentDatabase([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:25.477044 [ 2707 ] {d7b87570-4d59-4260-bd71-53a383a27830} executeQuery: Code: 42. DB::Exception: Number of arguments for function currentProfiles doesn't match: passed 3, should be 0: In scope SELECT currentProfiles([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT currentProfiles([], [], []);) 2025.09.25 21:10:25.482377 [ 2707 ] {d7b87570-4d59-4260-bd71-53a383a27830} TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentProfiles doesn't match: passed 3, should be 0: In scope SELECT currentProfiles([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:25.539303 [ 2707 ] {c7531fb9-f66e-41e2-a634-03a4221358eb} executeQuery: Code: 42. DB::Exception: Number of arguments for function currentRoles doesn't match: passed 3, should be 0: In scope SELECT currentRoles([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT currentRoles([], [], []);) 2025.09.25 21:10:25.545122 [ 2707 ] {c7531fb9-f66e-41e2-a634-03a4221358eb} TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentRoles doesn't match: passed 3, should be 0: In scope SELECT currentRoles([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:25.609024 [ 2707 ] {af6d724d-48dc-4c06-94de-125faee6f5da} executeQuery: Code: 42. DB::Exception: Number of arguments for function currentSchemas doesn't match: passed 3, should be 1: In scope SELECT currentSchemas([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT currentSchemas([], [], []);) 2025.09.25 21:10:25.617727 [ 2707 ] {af6d724d-48dc-4c06-94de-125faee6f5da} TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentSchemas doesn't match: passed 3, should be 1: In scope SELECT currentSchemas([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:25.726678 [ 2707 ] {e403c36f-f2c0-4697-82da-8a2e7a893b8b} executeQuery: Code: 42. DB::Exception: Number of arguments for function currentUser doesn't match: passed 3, should be 0: In scope SELECT currentUser([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT currentUser([], [], []);) 2025.09.25 21:10:25.734766 [ 2707 ] {e403c36f-f2c0-4697-82da-8a2e7a893b8b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentUser doesn't match: passed 3, should be 0: In scope SELECT currentUser([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:25.758456 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:25.834230 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:25.838678 [ 2707 ] {1995940c-8b79-492f-b974-2114b21a0ec2} executeQuery: Code: 42. DB::Exception: Number of arguments for function currentDatabase doesn't match: passed 3, should be 0: In scope SELECT current_database([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT current_database([], [], []);) 2025.09.25 21:10:25.844361 [ 2707 ] {1995940c-8b79-492f-b974-2114b21a0ec2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentDatabase doesn't match: passed 3, should be 0: In scope SELECT current_database([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:25.900588 [ 2707 ] {ff04c86c-a2db-499e-98e0-4d95b1ad6b4f} executeQuery: Code: 42. DB::Exception: Number of arguments for function today doesn't match: passed 3, should be 0: In scope SELECT current_date([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT current_date([], [], []);) 2025.09.25 21:10:25.922678 [ 2707 ] {ff04c86c-a2db-499e-98e0-4d95b1ad6b4f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function today doesn't match: passed 3, should be 0: In scope SELECT current_date([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:26.089057 [ 2707 ] {05ec273b-c6f6-4f43-8a8e-ba52d480ee60} executeQuery: Code: 42. DB::Exception: Number of arguments for function currentSchemas doesn't match: passed 3, should be 1: In scope SELECT current_schemas([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT current_schemas([], [], []);) 2025.09.25 21:10:26.099616 [ 2707 ] {05ec273b-c6f6-4f43-8a8e-ba52d480ee60} TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentSchemas doesn't match: passed 3, should be 1: In scope SELECT current_schemas([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:26.187398 [ 2707 ] {ceb35a66-d969-4a14-bb01-913a05145d13} executeQuery: Code: 42. DB::Exception: Arguments size of function now should be 0 or 1: In scope SELECT current_timestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT current_timestamp([], [], []);) 2025.09.25 21:10:26.198010 [ 2707 ] {ceb35a66-d969-4a14-bb01-913a05145d13} TCPHandler: Code: 42. DB::Exception: Arguments size of function now should be 0 or 1: In scope SELECT current_timestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:26.314328 [ 2707 ] {8d210429-70fa-4db4-9fd9-bae3d07a9285} executeQuery: Code: 42. DB::Exception: Number of arguments for function currentUser doesn't match: passed 3, should be 0: In scope SELECT current_user([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT current_user([], [], []);) 2025.09.25 21:10:26.327221 [ 2707 ] {8d210429-70fa-4db4-9fd9-bae3d07a9285} TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentUser doesn't match: passed 3, should be 0: In scope SELECT current_user([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:26.421032 [ 2707 ] {bbd8fe6f-d498-480a-a913-372f75afb58d} executeQuery: Code: 42. DB::Exception: Number of arguments for function cutFragment doesn't match: passed 3, should be 1: In scope SELECT cutFragment([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cutFragment([], [], []);) 2025.09.25 21:10:26.431814 [ 2707 ] {bbd8fe6f-d498-480a-a913-372f75afb58d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutFragment doesn't match: passed 3, should be 1: In scope SELECT cutFragment([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:26.573012 [ 2707 ] {08ad630d-1c11-4b57-adaf-b0cabeb4838e} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument 1 of function cutIPv6, expected FixedString(16): In scope SELECT cutIPv6([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cutIPv6([], [], []);) 2025.09.25 21:10:26.578298 [ 2707 ] {08ad630d-1c11-4b57-adaf-b0cabeb4838e} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument 1 of function cutIPv6, expected FixedString(16): In scope SELECT cutIPv6([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:26.661948 [ 2707 ] {781e6409-31ed-4a78-9041-ac4e45b4096a} executeQuery: Code: 42. DB::Exception: Number of arguments for function cutQueryString doesn't match: passed 3, should be 1: In scope SELECT cutQueryString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cutQueryString([], [], []);) 2025.09.25 21:10:26.670434 [ 2707 ] {781e6409-31ed-4a78-9041-ac4e45b4096a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutQueryString doesn't match: passed 3, should be 1: In scope SELECT cutQueryString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:26.674468 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:26.725838 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:26.793548 [ 2707 ] {90b29472-a58b-487b-9e74-cea474c8bd21} executeQuery: Code: 42. DB::Exception: Number of arguments for function cutQueryStringAndFragment doesn't match: passed 3, should be 1: In scope SELECT cutQueryStringAndFragment([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cutQueryStringAndFragment([], [], []);) 2025.09.25 21:10:26.801040 [ 2707 ] {90b29472-a58b-487b-9e74-cea474c8bd21} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutQueryStringAndFragment doesn't match: passed 3, should be 1: In scope SELECT cutQueryStringAndFragment([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:26.971628 [ 2707 ] {c2b60f32-ba86-42db-baca-efcfde7c8955} executeQuery: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomain doesn't match: passed 3, should be 1: In scope SELECT cutToFirstSignificantSubdomain([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cutToFirstSignificantSubdomain([], [], []);) 2025.09.25 21:10:26.975292 [ 2707 ] {c2b60f32-ba86-42db-baca-efcfde7c8955} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomain doesn't match: passed 3, should be 1: In scope SELECT cutToFirstSignificantSubdomain([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:27.048775 [ 2707 ] {0ba03dfc-6e48-4964-ad22-d01e4713b446} executeQuery: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainCustom doesn't match: passed 3, should be 2: In scope SELECT cutToFirstSignificantSubdomainCustom([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cutToFirstSignificantSubdomainCustom([], [], []);) 2025.09.25 21:10:27.058082 [ 2707 ] {0ba03dfc-6e48-4964-ad22-d01e4713b446} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainCustom doesn't match: passed 3, should be 2: In scope SELECT cutToFirstSignificantSubdomainCustom([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:27.159848 [ 2707 ] {1e000ec5-a640-4ef7-8e39-b05cab89232b} executeQuery: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainCustomRFC doesn't match: passed 3, should be 2: In scope SELECT cutToFirstSignificantSubdomainCustomRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cutToFirstSignificantSubdomainCustomRFC([], [], []);) 2025.09.25 21:10:27.168232 [ 2707 ] {1e000ec5-a640-4ef7-8e39-b05cab89232b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainCustomRFC doesn't match: passed 3, should be 2: In scope SELECT cutToFirstSignificantSubdomainCustomRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:27.277808 [ 2707 ] {83ec6fd3-1a8b-4c90-990e-a2eebd23129a} executeQuery: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainCustomWithWWW doesn't match: passed 3, should be 2: In scope SELECT cutToFirstSignificantSubdomainCustomWithWWW([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cutToFirstSignificantSubdomainCustomWithWWW([], [], []);) 2025.09.25 21:10:27.286584 [ 2707 ] {83ec6fd3-1a8b-4c90-990e-a2eebd23129a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainCustomWithWWW doesn't match: passed 3, should be 2: In scope SELECT cutToFirstSignificantSubdomainCustomWithWWW([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:27.378830 [ 2707 ] {768d3b59-bbbe-47a8-babd-b5fbbf054b46} executeQuery: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainCustomWithWWWRFC doesn't match: passed 3, should be 2: In scope SELECT cutToFirstSignificantSubdomainCustomWithWWWRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cutToFirstSignificantSubdomainCustomWithWWWRFC([], [], []);) 2025.09.25 21:10:27.388718 [ 2707 ] {768d3b59-bbbe-47a8-babd-b5fbbf054b46} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainCustomWithWWWRFC doesn't match: passed 3, should be 2: In scope SELECT cutToFirstSignificantSubdomainCustomWithWWWRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:27.552088 [ 3619 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:27.571327 [ 2707 ] {b082c2af-5c32-4f9d-a0df-a23afde1a255} executeQuery: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainRFC doesn't match: passed 3, should be 1: In scope SELECT cutToFirstSignificantSubdomainRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cutToFirstSignificantSubdomainRFC([], [], []);) 2025.09.25 21:10:27.613682 [ 2707 ] {b082c2af-5c32-4f9d-a0df-a23afde1a255} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainRFC doesn't match: passed 3, should be 1: In scope SELECT cutToFirstSignificantSubdomainRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:27.764798 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:27.772976 [ 2707 ] {ef2dcbbf-7530-4b2d-813c-43ef04ba0bc5} executeQuery: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainWithWWW doesn't match: passed 3, should be 1: In scope SELECT cutToFirstSignificantSubdomainWithWWW([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cutToFirstSignificantSubdomainWithWWW([], [], []);) 2025.09.25 21:10:27.778561 [ 2707 ] {ef2dcbbf-7530-4b2d-813c-43ef04ba0bc5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainWithWWW doesn't match: passed 3, should be 1: In scope SELECT cutToFirstSignificantSubdomainWithWWW([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:27.789929 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:27.990575 [ 2707 ] {3eb27729-7409-4c2b-b364-d157ed4ed6fa} executeQuery: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainWithWWWRFC doesn't match: passed 3, should be 1: In scope SELECT cutToFirstSignificantSubdomainWithWWWRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cutToFirstSignificantSubdomainWithWWWRFC([], [], []);) 2025.09.25 21:10:28.006228 [ 2707 ] {3eb27729-7409-4c2b-b364-d157ed4ed6fa} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutToFirstSignificantSubdomainWithWWWRFC doesn't match: passed 3, should be 1: In scope SELECT cutToFirstSignificantSubdomainWithWWWRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:28.095195 [ 2707 ] {0dd534e2-59cc-49e7-b8ed-1a45e92880d6} executeQuery: Code: 42. DB::Exception: Number of arguments for function cutURLParameter doesn't match: passed 3, should be 2: In scope SELECT cutURLParameter([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cutURLParameter([], [], []);) 2025.09.25 21:10:28.102024 [ 2707 ] {0dd534e2-59cc-49e7-b8ed-1a45e92880d6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutURLParameter doesn't match: passed 3, should be 2: In scope SELECT cutURLParameter([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:28.182782 [ 2707 ] {1142f98f-9c68-477c-9fd5-6865ac7237d8} executeQuery: Code: 42. DB::Exception: Number of arguments for function cutWWW doesn't match: passed 3, should be 1: In scope SELECT cutWWW([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT cutWWW([], [], []);) 2025.09.25 21:10:28.190506 [ 2707 ] {1142f98f-9c68-477c-9fd5-6865ac7237d8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function cutWWW doesn't match: passed 3, should be 1: In scope SELECT cutWWW([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:28.256999 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:28.268663 [ 2707 ] {6c2cf476-a02e-40b2-80f8-ee67e47c9873} executeQuery: Code: 42. DB::Exception: Number of arguments for function damerauLevenshteinDistance doesn't match: passed 3, should be 2: In scope SELECT damerauLevenshteinDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT damerauLevenshteinDistance([], [], []);) 2025.09.25 21:10:28.277326 [ 2707 ] {6c2cf476-a02e-40b2-80f8-ee67e47c9873} TCPHandler: Code: 42. DB::Exception: Number of arguments for function damerauLevenshteinDistance doesn't match: passed 3, should be 2: In scope SELECT damerauLevenshteinDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:28.353986 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:28.366966 [ 2707 ] {1e859206-e6ae-4c45-ad41-bd9f863cac44} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'unit' to function 'dateDiff'. Expected: String, got: Array(Nothing): In scope SELECT dateDiff([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dateDiff([], [], []);) 2025.09.25 21:10:28.386945 [ 2707 ] {1e859206-e6ae-4c45-ad41-bd9f863cac44} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'unit' to function 'dateDiff'. Expected: String, got: Array(Nothing): In scope SELECT dateDiff([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:28.459527 [ 2707 ] {fc473203-3054-48ed-952b-d30c68f74ed1} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of 1 argument of function dateName. Must be string: In scope SELECT dateName([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dateName([], [], []);) 2025.09.25 21:10:28.465254 [ 2707 ] {fc473203-3054-48ed-952b-d30c68f74ed1} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of 1 argument of function dateName. Must be string: In scope SELECT dateName([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:28.557361 [ 2707 ] {3316929c-931f-4460-8cb1-7489fc1b2d65} executeQuery: Code: 42. DB::Exception: Number of arguments for function dateTime64ToSnowflake doesn't match: passed 3, should be 1: In scope SELECT dateTime64ToSnowflake([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dateTime64ToSnowflake([], [], []);) 2025.09.25 21:10:28.562444 [ 2707 ] {3316929c-931f-4460-8cb1-7489fc1b2d65} TCPHandler: Code: 42. DB::Exception: Number of arguments for function dateTime64ToSnowflake doesn't match: passed 3, should be 1: In scope SELECT dateTime64ToSnowflake([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:28.621541 [ 2707 ] {2b6afde4-b3f0-4e4b-b14e-b203f2020f5b} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'dateTime64ToSnowflakeID'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT dateTime64ToSnowflakeID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dateTime64ToSnowflakeID([], [], []);) 2025.09.25 21:10:28.633230 [ 2707 ] {2b6afde4-b3f0-4e4b-b14e-b203f2020f5b} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'dateTime64ToSnowflakeID'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT dateTime64ToSnowflakeID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:28.766839 [ 2707 ] {5232cccf-7952-4b50-bc7f-58f7df6fd03b} executeQuery: Code: 42. DB::Exception: Number of arguments for function dateTimeToSnowflake doesn't match: passed 3, should be 1: In scope SELECT dateTimeToSnowflake([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dateTimeToSnowflake([], [], []);) 2025.09.25 21:10:28.782324 [ 2707 ] {5232cccf-7952-4b50-bc7f-58f7df6fd03b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function dateTimeToSnowflake doesn't match: passed 3, should be 1: In scope SELECT dateTimeToSnowflake([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:28.902827 [ 2707 ] {644fa03c-131d-4879-abe2-9ade8611aa7b} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'dateTimeToSnowflakeID'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT dateTimeToSnowflakeID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dateTimeToSnowflakeID([], [], []);) 2025.09.25 21:10:28.911119 [ 2707 ] {644fa03c-131d-4879-abe2-9ade8611aa7b} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'dateTimeToSnowflakeID'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT dateTimeToSnowflakeID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:29.037024 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:29.048433 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:29.074791 [ 2707 ] {02c32f1e-70c0-4a0d-b8bd-a4ca3973d4c6} executeQuery: Code: 43. DB::Exception: First argument for function dateTrunc must be constant string: name of datepart: In scope SELECT dateTrunc([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dateTrunc([], [], []);) 2025.09.25 21:10:29.092006 [ 2707 ] {02c32f1e-70c0-4a0d-b8bd-a4ca3973d4c6} TCPHandler: Code: 43. DB::Exception: First argument for function dateTrunc must be constant string: name of datepart: In scope SELECT dateTrunc([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:29.202004 [ 2707 ] {2e8bdba3-b669-4aad-9697-808ec8b36dda} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'unit' to function 'dateDiff'. Expected: String, got: Array(Nothing): In scope SELECT dateDiff([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT date_diff([], [], []);) 2025.09.25 21:10:29.281170 [ 2707 ] {2e8bdba3-b669-4aad-9697-808ec8b36dda} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'unit' to function 'dateDiff'. Expected: String, got: Array(Nothing): In scope SELECT dateDiff([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:29.352312 [ 2707 ] {83a05f58-d0a6-47c7-b53e-e05280dfcd2b} executeQuery: Code: 42. DB::Exception: Number of arguments for function decodeHTMLComponent doesn't match: passed 3, should be 1: In scope SELECT decodeHTMLComponent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT decodeHTMLComponent([], [], []);) 2025.09.25 21:10:29.358221 [ 2707 ] {83a05f58-d0a6-47c7-b53e-e05280dfcd2b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function decodeHTMLComponent doesn't match: passed 3, should be 1: In scope SELECT decodeHTMLComponent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:29.473707 [ 2707 ] {5bfb7dd5-58b7-4dfb-a65b-d19a881041c1} executeQuery: Code: 42. DB::Exception: Number of arguments for function decodeURLComponent doesn't match: passed 3, should be 1: In scope SELECT decodeURLComponent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT decodeURLComponent([], [], []);) 2025.09.25 21:10:29.494518 [ 2707 ] {5bfb7dd5-58b7-4dfb-a65b-d19a881041c1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function decodeURLComponent doesn't match: passed 3, should be 1: In scope SELECT decodeURLComponent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:29.587737 [ 2707 ] {957f2b33-ec05-4536-aeab-2eba905ec9bf} executeQuery: Code: 42. DB::Exception: Number of arguments for function decodeURLFormComponent doesn't match: passed 3, should be 1: In scope SELECT decodeURLFormComponent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT decodeURLFormComponent([], [], []);) 2025.09.25 21:10:29.593650 [ 2707 ] {957f2b33-ec05-4536-aeab-2eba905ec9bf} TCPHandler: Code: 42. DB::Exception: Number of arguments for function decodeURLFormComponent doesn't match: passed 3, should be 1: In scope SELECT decodeURLFormComponent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:29.683698 [ 2707 ] {938a123f-9319-47d7-857d-acc372e95605} executeQuery: Code: 42. DB::Exception: Number of arguments for function decodeXMLComponent doesn't match: passed 3, should be 1: In scope SELECT decodeXMLComponent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT decodeXMLComponent([], [], []);) 2025.09.25 21:10:29.696950 [ 2707 ] {938a123f-9319-47d7-857d-acc372e95605} TCPHandler: Code: 42. DB::Exception: Number of arguments for function decodeXMLComponent doesn't match: passed 3, should be 1: In scope SELECT decodeXMLComponent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:29.845107 [ 2707 ] {cc8174f4-021b-471d-b209-240b0faf7da4} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'mode' to function 'decrypt'. Expected: decryption mode string, got: Array(Nothing): In scope SELECT decrypt([], '[HIDDEN]'). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT decrypt([], '[HIDDEN]')) 2025.09.25 21:10:29.860089 [ 2707 ] {cc8174f4-021b-471d-b209-240b0faf7da4} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'mode' to function 'decrypt'. Expected: decryption mode string, got: Array(Nothing): In scope SELECT decrypt([], '[HIDDEN]'). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:29.954652 [ 2707 ] {acfdd90f-e174-41d1-b643-4e5672e713cd} executeQuery: Code: 42. DB::Exception: Number of arguments for function defaultProfiles doesn't match: passed 3, should be 0: In scope SELECT defaultProfiles([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT defaultProfiles([], [], []);) 2025.09.25 21:10:29.965167 [ 2707 ] {acfdd90f-e174-41d1-b643-4e5672e713cd} TCPHandler: Code: 42. DB::Exception: Number of arguments for function defaultProfiles doesn't match: passed 3, should be 0: In scope SELECT defaultProfiles([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:30.085054 [ 2707 ] {25714767-28e8-4630-95b1-56fb8bd38e8c} executeQuery: Code: 42. DB::Exception: Number of arguments for function defaultRoles doesn't match: passed 3, should be 0: In scope SELECT defaultRoles([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT defaultRoles([], [], []);) 2025.09.25 21:10:30.181073 [ 2707 ] {25714767-28e8-4630-95b1-56fb8bd38e8c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function defaultRoles doesn't match: passed 3, should be 0: In scope SELECT defaultRoles([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:30.243344 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:30.287616 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:30.309568 [ 2707 ] {893fa6f2-dc57-4183-8039-cd7fc54c8399} executeQuery: Code: 42. DB::Exception: Number of arguments for function defaultValueOfArgumentType doesn't match: passed 3, should be 1: In scope SELECT defaultValueOfArgumentType([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT defaultValueOfArgumentType([], [], []);) 2025.09.25 21:10:30.316346 [ 2707 ] {893fa6f2-dc57-4183-8039-cd7fc54c8399} TCPHandler: Code: 42. DB::Exception: Number of arguments for function defaultValueOfArgumentType doesn't match: passed 3, should be 1: In scope SELECT defaultValueOfArgumentType([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:30.576980 [ 2707 ] {99c23fb2-5b19-41d3-a5db-b34fdf3fbbcf} executeQuery: Code: 42. DB::Exception: Number of arguments for function defaultValueOfTypeName doesn't match: passed 3, should be 1: In scope SELECT defaultValueOfTypeName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT defaultValueOfTypeName([], [], []);) 2025.09.25 21:10:30.607268 [ 2707 ] {99c23fb2-5b19-41d3-a5db-b34fdf3fbbcf} TCPHandler: Code: 42. DB::Exception: Number of arguments for function defaultValueOfTypeName doesn't match: passed 3, should be 1: In scope SELECT defaultValueOfTypeName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:30.749900 [ 2707 ] {0de4e349-96f7-41db-b361-e354de1a0cdb} executeQuery: Code: 42. DB::Exception: Number of arguments for function degrees doesn't match: passed 3, should be 1: In scope SELECT degrees([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT degrees([], [], []);) 2025.09.25 21:10:30.755649 [ 2707 ] {0de4e349-96f7-41db-b361-e354de1a0cdb} TCPHandler: Code: 42. DB::Exception: Number of arguments for function degrees doesn't match: passed 3, should be 1: In scope SELECT degrees([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:30.833279 [ 2707 ] {eb5332b1-e890-475c-bf63-ca7c457767d5} executeQuery: Code: 42. DB::Exception: Incorrect number of arguments for aggregate function deltaSum. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT deltaSum([], [], []);) 2025.09.25 21:10:30.845908 [ 2707 ] {eb5332b1-e890-475c-bf63-ca7c457767d5} TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments for aggregate function deltaSum. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:30.922361 [ 2707 ] {945a4720-7302-4b92-ba2a-5ece7e4e8fb0} executeQuery: Code: 42. DB::Exception: Aggregate function deltaSumTimestamp requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT deltaSumTimestamp([], [], []);) 2025.09.25 21:10:30.947272 [ 2707 ] {945a4720-7302-4b92-ba2a-5ece7e4e8fb0} TCPHandler: Code: 42. DB::Exception: Aggregate function deltaSumTimestamp requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:31.086278 [ 2707 ] {5d699875-c3fe-4181-b64b-0c52fc772915} executeQuery: Code: 42. DB::Exception: Number of arguments for function demangle doesn't match: passed 3, should be 1: In scope SELECT demangle([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT demangle([], [], []);) 2025.09.25 21:10:31.098322 [ 2707 ] {5d699875-c3fe-4181-b64b-0c52fc772915} TCPHandler: Code: 42. DB::Exception: Number of arguments for function demangle doesn't match: passed 3, should be 1: In scope SELECT demangle([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:31.241822 [ 2707 ] {98ed86f1-21fd-44f4-882c-e8150ba3201c} executeQuery: Code: 36. DB::Exception: The function 'denseRank' can only be used as a window function, not as an aggregate function: While executing AggregatingTransform. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT denseRank([], [], []);) 2025.09.25 21:10:31.249953 [ 2707 ] {98ed86f1-21fd-44f4-882c-e8150ba3201c} TCPHandler: Code: 36. DB::Exception: The function 'denseRank' can only be used as a window function, not as an aggregate function: While executing AggregatingTransform. (BAD_ARGUMENTS) 2025.09.25 21:10:31.347460 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:31.381711 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:31.456118 [ 2707 ] {28b496b5-da6e-4ffb-8bd4-c3b4237d2ad8} executeQuery: Code: 36. DB::Exception: The function 'denseRank' can only be used as a window function, not as an aggregate function: While executing AggregatingTransform. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dense_rank([], [], []);) 2025.09.25 21:10:31.472547 [ 2707 ] {28b496b5-da6e-4ffb-8bd4-c3b4237d2ad8} TCPHandler: Code: 36. DB::Exception: The function 'denseRank' can only be used as a window function, not as an aggregate function: While executing AggregatingTransform. (BAD_ARGUMENTS) 2025.09.25 21:10:31.606139 [ 2707 ] {3b2b9f38-9157-4e63-b62b-bb6cf4bae376} executeQuery: Code: 344. DB::Exception: Natural language processing function 'detectCharset' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT detectCharset([], [], []);) 2025.09.25 21:10:31.612598 [ 2707 ] {3b2b9f38-9157-4e63-b62b-bb6cf4bae376} TCPHandler: Code: 344. DB::Exception: Natural language processing function 'detectCharset' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) 2025.09.25 21:10:31.749421 [ 2707 ] {a7ecd3c9-3315-40f3-836a-c556c996dbe0} executeQuery: Code: 344. DB::Exception: Natural language processing function 'detectLanguage' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT detectLanguage([], [], []);) 2025.09.25 21:10:31.770964 [ 2707 ] {a7ecd3c9-3315-40f3-836a-c556c996dbe0} TCPHandler: Code: 344. DB::Exception: Natural language processing function 'detectLanguage' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) 2025.09.25 21:10:31.876286 [ 2707 ] {556277e1-14a9-4854-b334-5b0b2a23328f} executeQuery: Code: 344. DB::Exception: Natural language processing function 'detectLanguageMixed' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT detectLanguageMixed([], [], []);) 2025.09.25 21:10:31.885007 [ 2707 ] {556277e1-14a9-4854-b334-5b0b2a23328f} TCPHandler: Code: 344. DB::Exception: Natural language processing function 'detectLanguageMixed' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) 2025.09.25 21:10:32.020653 [ 2707 ] {e8a3f298-213e-4316-b484-2ade8762de45} executeQuery: Code: 344. DB::Exception: Natural language processing function 'detectLanguageUnknown' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT detectLanguageUnknown([], [], []);) 2025.09.25 21:10:32.022958 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:32.055679 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:32.118341 [ 2707 ] {e8a3f298-213e-4316-b484-2ade8762de45} TCPHandler: Code: 344. DB::Exception: Natural language processing function 'detectLanguageUnknown' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) 2025.09.25 21:10:32.178815 [ 2707 ] {cfe1ce81-51d9-4a71-a405-50a85079586e} executeQuery: Code: 344. DB::Exception: Natural language processing function 'detectProgrammingLanguage' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT detectProgrammingLanguage([], [], []);) 2025.09.25 21:10:32.202355 [ 2707 ] {cfe1ce81-51d9-4a71-a405-50a85079586e} TCPHandler: Code: 344. DB::Exception: Natural language processing function 'detectProgrammingLanguage' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) 2025.09.25 21:10:32.339139 [ 2707 ] {3d0d696a-bb4d-44f5-9aa1-07d1072d1830} executeQuery: Code: 344. DB::Exception: Natural language processing function 'detectTonality' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT detectTonality([], [], []);) 2025.09.25 21:10:32.430132 [ 2707 ] {3d0d696a-bb4d-44f5-9aa1-07d1072d1830} TCPHandler: Code: 344. DB::Exception: Natural language processing function 'detectTonality' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) 2025.09.25 21:10:32.549574 [ 2707 ] {2e2ffed7-cc80-4621-86d2-18a4099a615d} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGet([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGet([], [], []);) 2025.09.25 21:10:32.554584 [ 2707 ] {2e2ffed7-cc80-4621-86d2-18a4099a615d} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGet([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:32.653871 [ 2707 ] {630d5799-b736-48bd-b1bc-c396176a70c3} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetAll, expected a const string.: In scope SELECT dictGetAll([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetAll([], [], []);) 2025.09.25 21:10:32.673452 [ 2707 ] {630d5799-b736-48bd-b1bc-c396176a70c3} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetAll, expected a const string.: In scope SELECT dictGetAll([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:32.758318 [ 2707 ] {8d3f9ff8-16b5-4820-9a31-a3ffcc6d5183} executeQuery: Code: 42. DB::Exception: Number of arguments for function dictGetChildren doesn't match: passed 3, should be 2: In scope SELECT dictGetChildren([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetChildren([], [], []);) 2025.09.25 21:10:32.763082 [ 2707 ] {8d3f9ff8-16b5-4820-9a31-a3ffcc6d5183} TCPHandler: Code: 42. DB::Exception: Number of arguments for function dictGetChildren doesn't match: passed 3, should be 2: In scope SELECT dictGetChildren([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:32.789934 [ 3682 ] {0e6967e6-52d6-4d89-b2d9-fa005d32d623} executeQuery: Code: 107. DB::Exception: File /var/lib/clickhouse/user_files/data.jsonl doesn't exist. (FILE_DOESNT_EXIST) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:49404) (comment: 02222_create_table_without_columns_metadata.sh) (in query: select * from test), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageFile.cpp:1604: DB::StorageFile::read(DB::QueryPlan&, std::vector> const&, std::shared_ptr const&, DB::SelectQueryInfo&, std::shared_ptr, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x000000002b79db62 5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:880: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr&, bool, bool) @ 0x0000000029469c4b 6. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1795: DB::buildJoinTreeQueryPlan(std::shared_ptr const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002945d99d 7. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000002942c986 8. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 9. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 13. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:32.804056 [ 3682 ] {0e6967e6-52d6-4d89-b2d9-fa005d32d623} TCPHandler: Code: 107. DB::Exception: File /var/lib/clickhouse/user_files/data.jsonl doesn't exist. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageFile.cpp:1604: DB::StorageFile::read(DB::QueryPlan&, std::vector> const&, std::shared_ptr const&, DB::SelectQueryInfo&, std::shared_ptr, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x000000002b79db62 5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:880: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr&, bool, bool) @ 0x0000000029469c4b 6. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1795: DB::buildJoinTreeQueryPlan(std::shared_ptr const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002945d99d 7. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000002942c986 8. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 9. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 13. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:32.847273 [ 2707 ] {9400a40e-d9af-4eef-a5fa-711580b1a68c} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetDate([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetDate([], [], []);) 2025.09.25 21:10:32.863101 [ 2707 ] {9400a40e-d9af-4eef-a5fa-711580b1a68c} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetDate([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:32.972014 [ 2707 ] {2b2c9896-02c1-429d-85a2-3e9d4af7c092} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetDateOrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetDateOrDefault([], [], []);) 2025.09.25 21:10:33.000550 [ 2707 ] {2b2c9896-02c1-429d-85a2-3e9d4af7c092} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetDateOrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:33.107673 [ 2707 ] {a62e8737-d20b-4137-986a-453a301fa3b6} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetDateTime([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetDateTime([], [], []);) 2025.09.25 21:10:33.115543 [ 2707 ] {a62e8737-d20b-4137-986a-453a301fa3b6} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetDateTime([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:33.218463 [ 2707 ] {60f34f8d-c623-4a00-bf0d-d4cc54196571} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetDateTimeOrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetDateTimeOrDefault([], [], []);) 2025.09.25 21:10:33.229037 [ 2707 ] {60f34f8d-c623-4a00-bf0d-d4cc54196571} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetDateTimeOrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:33.283995 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:33.430182 [ 2707 ] {a5db080e-be4c-4458-a05d-cb7ad0d5ca4f} executeQuery: Code: 43. DB::Exception: Illegal type of first argument of function dictGetDescendants. Expected const String. Actual type Array(Nothing): In scope SELECT dictGetDescendants([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetDescendants([], [], []);) 2025.09.25 21:10:33.431188 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:33.438139 [ 2707 ] {a5db080e-be4c-4458-a05d-cb7ad0d5ca4f} TCPHandler: Code: 43. DB::Exception: Illegal type of first argument of function dictGetDescendants. Expected const String. Actual type Array(Nothing): In scope SELECT dictGetDescendants([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:33.599368 [ 2707 ] {668d526f-89f3-4d9b-8b5b-30d8a50f691f} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetFloat32([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetFloat32([], [], []);) 2025.09.25 21:10:33.606305 [ 2707 ] {668d526f-89f3-4d9b-8b5b-30d8a50f691f} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetFloat32([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:33.678768 [ 2707 ] {846a863e-ce55-4d04-8433-186b987845f1} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetFloat32OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetFloat32OrDefault([], [], []);) 2025.09.25 21:10:33.694760 [ 2707 ] {846a863e-ce55-4d04-8433-186b987845f1} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetFloat32OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:33.765287 [ 2707 ] {7b13eefe-1d8e-4197-8b52-51ab57755496} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetFloat64([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetFloat64([], [], []);) 2025.09.25 21:10:33.770471 [ 2707 ] {7b13eefe-1d8e-4197-8b52-51ab57755496} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetFloat64([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:33.868070 [ 2707 ] {528f7aa3-9585-47a4-9806-419b4221b815} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetFloat64OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetFloat64OrDefault([], [], []);) 2025.09.25 21:10:33.878447 [ 2707 ] {528f7aa3-9585-47a4-9806-419b4221b815} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetFloat64OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:34.013006 [ 2707 ] {adf17a20-6fc1-4f0a-acf7-1565eeb289e6} executeQuery: Code: 42. DB::Exception: Number of arguments for function dictGetHierarchy doesn't match: passed 3, should be 2: In scope SELECT dictGetHierarchy([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetHierarchy([], [], []);) 2025.09.25 21:10:34.018783 [ 2707 ] {adf17a20-6fc1-4f0a-acf7-1565eeb289e6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function dictGetHierarchy doesn't match: passed 3, should be 2: In scope SELECT dictGetHierarchy([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:34.082029 [ 2707 ] {579d9b76-aba0-465c-b019-36daa35ed335} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetIPv4([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetIPv4([], [], []);) 2025.09.25 21:10:34.088885 [ 2707 ] {579d9b76-aba0-465c-b019-36daa35ed335} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetIPv4([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:34.165420 [ 2707 ] {67021e02-c7cf-481d-b324-685c4500a79c} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetIPv4OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetIPv4OrDefault([], [], []);) 2025.09.25 21:10:34.173819 [ 2707 ] {67021e02-c7cf-481d-b324-685c4500a79c} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetIPv4OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:34.223967 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:34.266634 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:34.294453 [ 2707 ] {130bce38-a3af-432d-99f1-dbfc8f5d31bd} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetIPv6([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetIPv6([], [], []);) 2025.09.25 21:10:34.300727 [ 2707 ] {130bce38-a3af-432d-99f1-dbfc8f5d31bd} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetIPv6([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:34.424344 [ 2707 ] {9f1ab992-9543-4689-90ba-5719ae038607} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetIPv6OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetIPv6OrDefault([], [], []);) 2025.09.25 21:10:34.435961 [ 2707 ] {9f1ab992-9543-4689-90ba-5719ae038607} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetIPv6OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:34.535268 [ 2707 ] {0c361040-0da4-4a41-b916-5cb2affee929} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetInt16([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetInt16([], [], []);) 2025.09.25 21:10:34.540463 [ 2707 ] {0c361040-0da4-4a41-b916-5cb2affee929} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetInt16([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:34.641184 [ 2707 ] {fb9e7c64-c7c3-484f-9667-934ed1158530} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetInt16OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetInt16OrDefault([], [], []);) 2025.09.25 21:10:34.648540 [ 2707 ] {fb9e7c64-c7c3-484f-9667-934ed1158530} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetInt16OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:34.724546 [ 2707 ] {d333d748-4b83-4b2f-a026-852d95c1322f} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetInt32([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetInt32([], [], []);) 2025.09.25 21:10:34.730734 [ 2707 ] {d333d748-4b83-4b2f-a026-852d95c1322f} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetInt32([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:34.773600 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:34.794054 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:34.891314 [ 2707 ] {b639e1fc-020e-4c9e-95c2-d708af6d3146} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetInt32OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetInt32OrDefault([], [], []);) 2025.09.25 21:10:34.914018 [ 2707 ] {b639e1fc-020e-4c9e-95c2-d708af6d3146} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetInt32OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:35.028960 [ 2707 ] {49460b94-861e-445f-97ab-afe757190088} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetInt64([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetInt64([], [], []);) 2025.09.25 21:10:35.034760 [ 2707 ] {49460b94-861e-445f-97ab-afe757190088} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetInt64([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:35.128168 [ 2707 ] {8029acfd-9910-435c-abb6-f51a49b2fc50} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetInt64OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetInt64OrDefault([], [], []);) 2025.09.25 21:10:35.134774 [ 2707 ] {8029acfd-9910-435c-abb6-f51a49b2fc50} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetInt64OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:35.228544 [ 2707 ] {4d77c15c-bd0b-4e52-a603-c5c632c4b5f3} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetInt8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetInt8([], [], []);) 2025.09.25 21:10:35.237717 [ 2707 ] {4d77c15c-bd0b-4e52-a603-c5c632c4b5f3} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetInt8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:35.319791 [ 2707 ] {f0e07e29-a20d-4a0f-a207-e966c723f128} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetInt8OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetInt8OrDefault([], [], []);) 2025.09.25 21:10:35.337101 [ 2707 ] {f0e07e29-a20d-4a0f-a207-e966c723f128} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetInt8OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:35.470527 [ 2707 ] {d26920f6-26d0-4928-a2e1-c668639966bc} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetOrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetOrDefault([], [], []);) 2025.09.25 21:10:35.474920 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:35.476893 [ 2707 ] {d26920f6-26d0-4928-a2e1-c668639966bc} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetOrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:35.526348 [ 2707 ] {9e7e7fc4-a0c2-46ea-84d5-d3487905a9be} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetOrNull([], [], []);) 2025.09.25 21:10:35.532532 [ 2707 ] {9e7e7fc4-a0c2-46ea-84d5-d3487905a9be} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:35.605664 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:35.661407 [ 2707 ] {e2a21334-49e1-473d-8ff3-7d13fc609674} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetString([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetString([], [], []);) 2025.09.25 21:10:35.675902 [ 2707 ] {e2a21334-49e1-473d-8ff3-7d13fc609674} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetString([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:35.893253 [ 2707 ] {88f9c346-cdd1-41ff-a633-10d4a05a435c} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetStringOrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetStringOrDefault([], [], []);) 2025.09.25 21:10:35.899651 [ 2707 ] {88f9c346-cdd1-41ff-a633-10d4a05a435c} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetStringOrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:35.967514 [ 2707 ] {4a666c9f-39aa-4fbd-b1f2-c21494ce5040} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetUInt16([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetUInt16([], [], []);) 2025.09.25 21:10:35.973368 [ 2707 ] {4a666c9f-39aa-4fbd-b1f2-c21494ce5040} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetUInt16([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:36.040614 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:36.054242 [ 2707 ] {13f59dba-431b-408a-8649-25fc0b6fa50f} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetUInt16OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetUInt16OrDefault([], [], []);) 2025.09.25 21:10:36.058515 [ 2707 ] {13f59dba-431b-408a-8649-25fc0b6fa50f} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetUInt16OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:36.166429 [ 2707 ] {3b1fdc83-cf7d-4fb3-89f3-0b3384d72794} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetUInt32([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetUInt32([], [], []);) 2025.09.25 21:10:36.172311 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:36.175854 [ 2707 ] {3b1fdc83-cf7d-4fb3-89f3-0b3384d72794} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetUInt32([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:36.372138 [ 2707 ] {63bb2935-a794-446a-8271-71d7c69b7b3d} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetUInt32OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetUInt32OrDefault([], [], []);) 2025.09.25 21:10:36.384748 [ 2707 ] {63bb2935-a794-446a-8271-71d7c69b7b3d} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetUInt32OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:36.495659 [ 2707 ] {fde37adf-156a-4f87-900a-3ff17b1344ec} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetUInt64([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetUInt64([], [], []);) 2025.09.25 21:10:36.508326 [ 2707 ] {fde37adf-156a-4f87-900a-3ff17b1344ec} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetUInt64([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:36.636691 [ 2707 ] {b6c1c3c6-eb6c-4f95-a2cd-53d2e5e5d475} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetUInt64OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetUInt64OrDefault([], [], []);) 2025.09.25 21:10:36.647261 [ 2707 ] {b6c1c3c6-eb6c-4f95-a2cd-53d2e5e5d475} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetUInt64OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:36.732625 [ 2707 ] {19b39e4e-dfb0-4414-9275-8b1e0ee4b73d} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetUInt8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetUInt8([], [], []);) 2025.09.25 21:10:36.738767 [ 2707 ] {19b39e4e-dfb0-4414-9275-8b1e0ee4b73d} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetUInt8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:36.838432 [ 2707 ] {4b080c68-4e52-4745-b407-32de3bb33770} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetUInt8OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetUInt8OrDefault([], [], []);) 2025.09.25 21:10:36.845342 [ 2707 ] {4b080c68-4e52-4745-b407-32de3bb33770} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetUInt8OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:36.865710 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:36.903316 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:37.024119 [ 2707 ] {dd4c32dd-8830-41a7-a932-f31560e85e16} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetUUID([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetUUID([], [], []);) 2025.09.25 21:10:37.040626 [ 2707 ] {dd4c32dd-8830-41a7-a932-f31560e85e16} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGet, expected a const string.: In scope SELECT dictGetUUID([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:37.184553 [ 2707 ] {6c3887c1-8713-4371-82ca-391b842cc280} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetUUIDOrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictGetUUIDOrDefault([], [], []);) 2025.09.25 21:10:37.189977 [ 2707 ] {6c3887c1-8713-4371-82ca-391b842cc280} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictGetOrDefault, expected a const string.: In scope SELECT dictGetUUIDOrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:37.260830 [ 2707 ] {fd557112-2d4b-49e8-b2de-9210296cb8b9} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictHas, expected a string: In scope SELECT dictHas([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictHas([], [], []);) 2025.09.25 21:10:37.272541 [ 2707 ] {fd557112-2d4b-49e8-b2de-9210296cb8b9} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function dictHas, expected a string: In scope SELECT dictHas([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:37.307937 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:37.345249 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:37.373044 [ 2707 ] {55f2fa9a-dae0-4ccc-b418-116b27d6322a} executeQuery: Code: 43. DB::Exception: Illegal type of first argument of function dictIsIn. Expected String. Actual type Array(Nothing): In scope SELECT dictIsIn([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dictIsIn([], [], []);) 2025.09.25 21:10:37.395447 [ 2707 ] {55f2fa9a-dae0-4ccc-b418-116b27d6322a} TCPHandler: Code: 43. DB::Exception: Illegal type of first argument of function dictIsIn. Expected String. Actual type Array(Nothing): In scope SELECT dictIsIn([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:37.461765 [ 2707 ] {c537c0a5-942f-4102-902b-1aa271464855} executeQuery: Code: 42. DB::Exception: Number of arguments for function displayName doesn't match: passed 3, should be 0: In scope SELECT displayName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT displayName([], [], []);) 2025.09.25 21:10:37.466128 [ 2707 ] {c537c0a5-942f-4102-902b-1aa271464855} TCPHandler: Code: 42. DB::Exception: Number of arguments for function displayName doesn't match: passed 3, should be 0: In scope SELECT displayName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:37.560932 [ 2707 ] {55e0894a-7e5d-46fe-bd36-086d924ed2cf} executeQuery: Code: 42. DB::Exception: Number of arguments for function L1Distance doesn't match: passed 3, should be 2: In scope SELECT distanceL1([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT distanceL1([], [], []);) 2025.09.25 21:10:37.566338 [ 2707 ] {55e0894a-7e5d-46fe-bd36-086d924ed2cf} TCPHandler: Code: 42. DB::Exception: Number of arguments for function L1Distance doesn't match: passed 3, should be 2: In scope SELECT distanceL1([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:37.804135 [ 2707 ] {3f87af39-9d34-47cf-b056-c9c54c506805} executeQuery: Code: 42. DB::Exception: Number of arguments for function L2Distance doesn't match: passed 3, should be 2: In scope SELECT distanceL2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT distanceL2([], [], []);) 2025.09.25 21:10:37.815353 [ 2707 ] {3f87af39-9d34-47cf-b056-c9c54c506805} TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2Distance doesn't match: passed 3, should be 2: In scope SELECT distanceL2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:37.924987 [ 2707 ] {bbc9adfe-0bd5-462e-ba50-55d34a9c0c28} executeQuery: Code: 42. DB::Exception: Number of arguments for function L2SquaredDistance doesn't match: passed 3, should be 2: In scope SELECT distanceL2Squared([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT distanceL2Squared([], [], []);) 2025.09.25 21:10:37.950335 [ 2707 ] {bbc9adfe-0bd5-462e-ba50-55d34a9c0c28} TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2SquaredDistance doesn't match: passed 3, should be 2: In scope SELECT distanceL2Squared([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:38.035050 [ 2707 ] {9f4708d5-8abd-4d4f-ba95-d00651995266} executeQuery: Code: 42. DB::Exception: Number of arguments for function LinfDistance doesn't match: passed 3, should be 2: In scope SELECT distanceLinf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT distanceLinf([], [], []);) 2025.09.25 21:10:38.040527 [ 2707 ] {9f4708d5-8abd-4d4f-ba95-d00651995266} TCPHandler: Code: 42. DB::Exception: Number of arguments for function LinfDistance doesn't match: passed 3, should be 2: In scope SELECT distanceLinf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:38.162804 [ 2707 ] {dba20654-765b-4b9c-9f9d-4bd0b75dc10d} executeQuery: Code: 43. DB::Exception: Arguments of function arrayLpDistance has nested type Nothing. Supported types: UInt8, UInt16, UInt32, UInt64, Int8, Int16, Int32, Int64, Float32, Float64.: In scope SELECT distanceLp([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT distanceLp([], [], []);) 2025.09.25 21:10:38.168884 [ 2707 ] {dba20654-765b-4b9c-9f9d-4bd0b75dc10d} TCPHandler: Code: 43. DB::Exception: Arguments of function arrayLpDistance has nested type Nothing. Supported types: UInt8, UInt16, UInt32, UInt64, Int8, Int16, Int32, Int64, Float32, Float64.: In scope SELECT distanceLp([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:38.236712 [ 2707 ] {89f961b8-b3a1-44b5-98c1-3d59d9439215} executeQuery: Code: 42. DB::Exception: Number of arguments for function divide doesn't match: passed 3, should be 2: In scope SELECT divide([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT divide([], [], []);) 2025.09.25 21:10:38.244741 [ 2707 ] {89f961b8-b3a1-44b5-98c1-3d59d9439215} TCPHandler: Code: 42. DB::Exception: Number of arguments for function divide doesn't match: passed 3, should be 2: In scope SELECT divide([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:38.332086 [ 2707 ] {8a234127-b1aa-4bff-911e-65d1ffed174a} executeQuery: Code: 43. DB::Exception: Arguments for divideDecimal function must be Decimal: In scope SELECT divideDecimal([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT divideDecimal([], [], []);) 2025.09.25 21:10:38.343784 [ 2707 ] {8a234127-b1aa-4bff-911e-65d1ffed174a} TCPHandler: Code: 43. DB::Exception: Arguments for divideDecimal function must be Decimal: In scope SELECT divideDecimal([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:38.441293 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:38.446417 [ 2707 ] {b60211bd-1426-4f6a-857d-83573070fc81} executeQuery: Code: 42. DB::Exception: Number of arguments for function domain doesn't match: passed 3, should be 1: In scope SELECT domain([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT domain([], [], []);) 2025.09.25 21:10:38.463431 [ 2707 ] {b60211bd-1426-4f6a-857d-83573070fc81} TCPHandler: Code: 42. DB::Exception: Number of arguments for function domain doesn't match: passed 3, should be 1: In scope SELECT domain([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:38.481513 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:38.521554 [ 2707 ] {1d3c1004-0bf8-42e0-8c89-6e12af927590} executeQuery: Code: 42. DB::Exception: Number of arguments for function domainRFC doesn't match: passed 3, should be 1: In scope SELECT domainRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT domainRFC([], [], []);) 2025.09.25 21:10:38.530162 [ 2707 ] {1d3c1004-0bf8-42e0-8c89-6e12af927590} TCPHandler: Code: 42. DB::Exception: Number of arguments for function domainRFC doesn't match: passed 3, should be 1: In scope SELECT domainRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:38.686632 [ 2707 ] {5b383263-6196-4256-b833-59df9d47c088} executeQuery: Code: 42. DB::Exception: Number of arguments for function domainWithoutWWW doesn't match: passed 3, should be 1: In scope SELECT domainWithoutWWW([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT domainWithoutWWW([], [], []);) 2025.09.25 21:10:38.692273 [ 2707 ] {5b383263-6196-4256-b833-59df9d47c088} TCPHandler: Code: 42. DB::Exception: Number of arguments for function domainWithoutWWW doesn't match: passed 3, should be 1: In scope SELECT domainWithoutWWW([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:38.772750 [ 2707 ] {d210c6fb-0586-492b-a416-5eaed1644912} executeQuery: Code: 42. DB::Exception: Number of arguments for function domainWithoutWWWRFC doesn't match: passed 3, should be 1: In scope SELECT domainWithoutWWWRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT domainWithoutWWWRFC([], [], []);) 2025.09.25 21:10:38.796145 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:38.843468 [ 2707 ] {d210c6fb-0586-492b-a416-5eaed1644912} TCPHandler: Code: 42. DB::Exception: Number of arguments for function domainWithoutWWWRFC doesn't match: passed 3, should be 1: In scope SELECT domainWithoutWWWRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:38.871445 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:38.931601 [ 2707 ] {b2cd34a8-5efe-4c0a-b724-c8de7be0184a} executeQuery: Code: 42. DB::Exception: Number of arguments for function dotProduct doesn't match: passed 3, should be 2: In scope SELECT dotProduct([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dotProduct([], [], []);) 2025.09.25 21:10:38.937800 [ 2707 ] {b2cd34a8-5efe-4c0a-b724-c8de7be0184a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function dotProduct doesn't match: passed 3, should be 2: In scope SELECT dotProduct([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:38.994079 [ 2707 ] {b3a14a34-7818-478b-81b9-c0d6dd99a330} executeQuery: Code: 42. DB::Exception: Number of arguments for function dumpColumnStructure doesn't match: passed 3, should be 1: In scope SELECT dumpColumnStructure([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dumpColumnStructure([], [], []);) 2025.09.25 21:10:38.998377 [ 2707 ] {b3a14a34-7818-478b-81b9-c0d6dd99a330} TCPHandler: Code: 42. DB::Exception: Number of arguments for function dumpColumnStructure doesn't match: passed 3, should be 1: In scope SELECT dumpColumnStructure([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:39.086073 [ 2707 ] {cab8892f-3748-4c0a-bf83-0435de38374e} executeQuery: Code: 42. DB::Exception: Number of arguments for function dynamicElement doesn't match: passed 3, should be 2: In scope SELECT dynamicElement([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dynamicElement([], [], []);) 2025.09.25 21:10:39.098863 [ 2707 ] {cab8892f-3748-4c0a-bf83-0435de38374e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function dynamicElement doesn't match: passed 3, should be 2: In scope SELECT dynamicElement([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:39.178692 [ 2707 ] {510d88dc-fcaa-45d4-a363-66d11a873c34} executeQuery: Code: 42. DB::Exception: Number of arguments for function dynamicType doesn't match: passed 3, should be 1: In scope SELECT dynamicType([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT dynamicType([], [], []);) 2025.09.25 21:10:39.199085 [ 2707 ] {510d88dc-fcaa-45d4-a363-66d11a873c34} TCPHandler: Code: 42. DB::Exception: Number of arguments for function dynamicType doesn't match: passed 3, should be 1: In scope SELECT dynamicType([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:39.439419 [ 2707 ] {c7591c63-2d0c-4384-be58-edf911001876} executeQuery: Code: 42. DB::Exception: Number of arguments for function e doesn't match: passed 3, should be 0: In scope SELECT e([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT e([], [], []);) 2025.09.25 21:10:39.449065 [ 2707 ] {c7591c63-2d0c-4384-be58-edf911001876} TCPHandler: Code: 42. DB::Exception: Number of arguments for function e doesn't match: passed 3, should be 0: In scope SELECT e([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:39.559962 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:39.584932 [ 2707 ] {68c7d9cc-7441-4d0d-80a4-7774dc3d9c7f} executeQuery: Code: 42. DB::Exception: Number of arguments for function editDistance doesn't match: passed 3, should be 2: In scope SELECT editDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT editDistance([], [], []);) 2025.09.25 21:10:39.594532 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:39.598839 [ 2707 ] {68c7d9cc-7441-4d0d-80a4-7774dc3d9c7f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function editDistance doesn't match: passed 3, should be 2: In scope SELECT editDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:39.727397 [ 2707 ] {242e14d3-db76-4ec8-b17e-e672038487c9} executeQuery: Code: 42. DB::Exception: Number of arguments for function editDistanceUTF8 doesn't match: passed 3, should be 2: In scope SELECT editDistanceUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT editDistanceUTF8([], [], []);) 2025.09.25 21:10:39.731734 [ 2707 ] {242e14d3-db76-4ec8-b17e-e672038487c9} TCPHandler: Code: 42. DB::Exception: Number of arguments for function editDistanceUTF8 doesn't match: passed 3, should be 2: In scope SELECT editDistanceUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:39.830129 [ 2707 ] {1d30ae74-6224-42d9-9229-063bd4bfe39c} executeQuery: Code: 42. DB::Exception: Number of arguments for function empty doesn't match: passed 3, should be 1: In scope SELECT empty([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT empty([], [], []);) 2025.09.25 21:10:39.837122 [ 2707 ] {1d30ae74-6224-42d9-9229-063bd4bfe39c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function empty doesn't match: passed 3, should be 1: In scope SELECT empty([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:39.950276 [ 2707 ] {17dec187-06a5-45e6-84b3-bad45e99c177} executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayDate doesn't match: passed 3, should be 0: In scope SELECT emptyArrayDate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT emptyArrayDate([], [], []);) 2025.09.25 21:10:39.956927 [ 2707 ] {17dec187-06a5-45e6-84b3-bad45e99c177} TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayDate doesn't match: passed 3, should be 0: In scope SELECT emptyArrayDate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:40.150913 [ 2707 ] {08f19db5-16a4-4c1b-ae83-79fe4b0bacf1} executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayDateTime doesn't match: passed 3, should be 0: In scope SELECT emptyArrayDateTime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT emptyArrayDateTime([], [], []);) 2025.09.25 21:10:40.156195 [ 2707 ] {08f19db5-16a4-4c1b-ae83-79fe4b0bacf1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayDateTime doesn't match: passed 3, should be 0: In scope SELECT emptyArrayDateTime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:40.280387 [ 2707 ] {b83afbbf-5fe0-42a0-84fa-5848273f3b0a} executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayFloat32 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayFloat32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT emptyArrayFloat32([], [], []);) 2025.09.25 21:10:40.290599 [ 2707 ] {b83afbbf-5fe0-42a0-84fa-5848273f3b0a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayFloat32 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayFloat32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:40.352085 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:40.367877 [ 2707 ] {10fff93f-f9c4-4f85-b197-af048555ccfc} executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayFloat64 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayFloat64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT emptyArrayFloat64([], [], []);) 2025.09.25 21:10:40.371753 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:40.376468 [ 2707 ] {10fff93f-f9c4-4f85-b197-af048555ccfc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayFloat64 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayFloat64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:40.461100 [ 2707 ] {cdedf0d5-46d5-4745-907e-45120737451c} executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayInt16 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayInt16([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT emptyArrayInt16([], [], []);) 2025.09.25 21:10:40.465024 [ 2707 ] {cdedf0d5-46d5-4745-907e-45120737451c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayInt16 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayInt16([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:40.720511 [ 2707 ] {cb7fa382-3bd1-4775-960a-fe5d6b9e7471} executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayInt32 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayInt32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT emptyArrayInt32([], [], []);) 2025.09.25 21:10:40.729137 [ 2707 ] {cb7fa382-3bd1-4775-960a-fe5d6b9e7471} TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayInt32 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayInt32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:40.826476 [ 2707 ] {f74d761c-5619-4a8f-8e42-2f37949314b4} executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayInt64 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayInt64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT emptyArrayInt64([], [], []);) 2025.09.25 21:10:40.831496 [ 2707 ] {f74d761c-5619-4a8f-8e42-2f37949314b4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayInt64 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayInt64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:40.915668 [ 2707 ] {fce9c651-6313-4471-82c7-ac6ce3f1b68d} executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayInt8 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayInt8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT emptyArrayInt8([], [], []);) 2025.09.25 21:10:40.923438 [ 2707 ] {fce9c651-6313-4471-82c7-ac6ce3f1b68d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayInt8 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayInt8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:41.085102 [ 2707 ] {8196c24f-9edc-4327-92a6-43a626297deb} executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayString doesn't match: passed 3, should be 0: In scope SELECT emptyArrayString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT emptyArrayString([], [], []);) 2025.09.25 21:10:41.088883 [ 2707 ] {8196c24f-9edc-4327-92a6-43a626297deb} TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayString doesn't match: passed 3, should be 0: In scope SELECT emptyArrayString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:41.183580 [ 2707 ] {516c870f-41c7-4408-a070-368c03ab1d23} executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayToSingle doesn't match: passed 3, should be 1: In scope SELECT emptyArrayToSingle([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT emptyArrayToSingle([], [], []);) 2025.09.25 21:10:41.192671 [ 2707 ] {516c870f-41c7-4408-a070-368c03ab1d23} TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayToSingle doesn't match: passed 3, should be 1: In scope SELECT emptyArrayToSingle([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:41.193325 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:41.251896 [ 2707 ] {93566fe4-ccec-44a5-90bc-dfb1c9b68742} executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayUInt16 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayUInt16([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT emptyArrayUInt16([], [], []);) 2025.09.25 21:10:41.254936 [ 2707 ] {93566fe4-ccec-44a5-90bc-dfb1c9b68742} TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayUInt16 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayUInt16([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:41.270329 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:41.290090 [ 2707 ] {0089189b-10c3-48cb-a1c8-3b8e7dac0942} executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayUInt32 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayUInt32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT emptyArrayUInt32([], [], []);) 2025.09.25 21:10:41.293126 [ 2707 ] {0089189b-10c3-48cb-a1c8-3b8e7dac0942} TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayUInt32 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayUInt32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:41.344985 [ 2707 ] {ca4fe277-dc97-4d2a-b57d-5de470eaae6f} executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayUInt64 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayUInt64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT emptyArrayUInt64([], [], []);) 2025.09.25 21:10:41.350552 [ 2707 ] {ca4fe277-dc97-4d2a-b57d-5de470eaae6f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayUInt64 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayUInt64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:41.407671 [ 2707 ] {c5aa55f2-e7f9-41d5-8ef1-3655e0a00f0e} executeQuery: Code: 42. DB::Exception: Number of arguments for function emptyArrayUInt8 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayUInt8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT emptyArrayUInt8([], [], []);) 2025.09.25 21:10:41.413387 [ 2707 ] {c5aa55f2-e7f9-41d5-8ef1-3655e0a00f0e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function emptyArrayUInt8 doesn't match: passed 3, should be 0: In scope SELECT emptyArrayUInt8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:41.476389 [ 2707 ] {300d5e40-8fa3-495b-bea7-73b33e20cbb6} executeQuery: Code: 42. DB::Exception: Number of arguments for function enabledProfiles doesn't match: passed 3, should be 0: In scope SELECT enabledProfiles([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT enabledProfiles([], [], []);) 2025.09.25 21:10:41.486022 [ 2707 ] {300d5e40-8fa3-495b-bea7-73b33e20cbb6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function enabledProfiles doesn't match: passed 3, should be 0: In scope SELECT enabledProfiles([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:41.533931 [ 2707 ] {5ff5edc6-c781-4d82-8cd8-842b7eb4bdd7} executeQuery: Code: 42. DB::Exception: Number of arguments for function enabledRoles doesn't match: passed 3, should be 0: In scope SELECT enabledRoles([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT enabledRoles([], [], []);) 2025.09.25 21:10:41.537947 [ 2707 ] {5ff5edc6-c781-4d82-8cd8-842b7eb4bdd7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function enabledRoles doesn't match: passed 3, should be 0: In scope SELECT enabledRoles([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:41.591014 [ 2707 ] {8d7bd5ab-4bf3-4d95-a4d0-3897382e6202} executeQuery: Code: 42. DB::Exception: Number of arguments for function encodeURLComponent doesn't match: passed 3, should be 1: In scope SELECT encodeURLComponent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT encodeURLComponent([], [], []);) 2025.09.25 21:10:41.597518 [ 2707 ] {8d7bd5ab-4bf3-4d95-a4d0-3897382e6202} TCPHandler: Code: 42. DB::Exception: Number of arguments for function encodeURLComponent doesn't match: passed 3, should be 1: In scope SELECT encodeURLComponent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:41.671406 [ 2707 ] {7574486c-885c-4a07-9575-e0adc321b3f7} executeQuery: Code: 42. DB::Exception: Number of arguments for function encodeURLFormComponent doesn't match: passed 3, should be 1: In scope SELECT encodeURLFormComponent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT encodeURLFormComponent([], [], []);) 2025.09.25 21:10:41.675889 [ 2707 ] {7574486c-885c-4a07-9575-e0adc321b3f7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function encodeURLFormComponent doesn't match: passed 3, should be 1: In scope SELECT encodeURLFormComponent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:41.727306 [ 2707 ] {a0560e3a-dc49-4bf6-9442-b7cb6f85a20f} executeQuery: Code: 42. DB::Exception: Number of arguments for function encodeXMLComponent doesn't match: passed 3, should be 1: In scope SELECT encodeXMLComponent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT encodeXMLComponent([], [], []);) 2025.09.25 21:10:41.770949 [ 2707 ] {a0560e3a-dc49-4bf6-9442-b7cb6f85a20f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function encodeXMLComponent doesn't match: passed 3, should be 1: In scope SELECT encodeXMLComponent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:41.821403 [ 2707 ] {a8ffc22c-b1b1-4e9d-b470-623b94b8ede6} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'mode' to function 'encrypt'. Expected: encryption mode string, got: Array(Nothing): In scope SELECT encrypt([], '[HIDDEN]'). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT encrypt([], '[HIDDEN]')) 2025.09.25 21:10:41.826882 [ 2707 ] {a8ffc22c-b1b1-4e9d-b470-623b94b8ede6} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'mode' to function 'encrypt'. Expected: encryption mode string, got: Array(Nothing): In scope SELECT encrypt([], '[HIDDEN]'). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:41.927668 [ 2707 ] {7cf8e478-81d4-4534-890b-f543acd23279} executeQuery: Code: 42. DB::Exception: Number of arguments for function endsWith doesn't match: passed 3, should be 2: In scope SELECT endsWith([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT endsWith([], [], []);) 2025.09.25 21:10:41.933271 [ 2707 ] {7cf8e478-81d4-4534-890b-f543acd23279} TCPHandler: Code: 42. DB::Exception: Number of arguments for function endsWith doesn't match: passed 3, should be 2: In scope SELECT endsWith([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:42.000465 [ 2707 ] {da006334-0c96-4875-9cfd-6d5c068761ba} executeQuery: Code: 42. DB::Exception: Number of arguments for function endsWithUTF8 doesn't match: passed 3, should be 2: In scope SELECT endsWithUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT endsWithUTF8([], [], []);) 2025.09.25 21:10:42.008714 [ 2707 ] {da006334-0c96-4875-9cfd-6d5c068761ba} TCPHandler: Code: 42. DB::Exception: Number of arguments for function endsWithUTF8 doesn't match: passed 3, should be 2: In scope SELECT endsWithUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:42.281579 [ 2707 ] {b9f02d79-eeb4-42af-bd99-864c2d88da8d} executeQuery: Code: 42. DB::Exception: Number of arguments for function equals doesn't match: passed 3, should be 2: In scope SELECT equals([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT equals([], [], []);) 2025.09.25 21:10:42.289780 [ 2707 ] {b9f02d79-eeb4-42af-bd99-864c2d88da8d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function equals doesn't match: passed 3, should be 2: In scope SELECT equals([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:42.308725 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:42.324790 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:42.510645 [ 2707 ] {9e641d5a-47c6-41c4-8c79-95b558a456c4} executeQuery: Code: 42. DB::Exception: Number of arguments for function erf doesn't match: passed 3, should be 1: In scope SELECT erf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT erf([], [], []);) 2025.09.25 21:10:42.527531 [ 2707 ] {9e641d5a-47c6-41c4-8c79-95b558a456c4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function erf doesn't match: passed 3, should be 1: In scope SELECT erf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:42.586944 [ 2707 ] {f6b6f841-ebac-4cd3-aecb-f1fd95f5c13a} executeQuery: Code: 42. DB::Exception: Number of arguments for function erfc doesn't match: passed 3, should be 1: In scope SELECT erfc([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT erfc([], [], []);) 2025.09.25 21:10:42.592018 [ 2707 ] {f6b6f841-ebac-4cd3-aecb-f1fd95f5c13a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function erfc doesn't match: passed 3, should be 1: In scope SELECT erfc([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:42.692688 [ 2707 ] {ce856fc1-d87f-4349-bd34-5a38165b0718} executeQuery: Code: 42. DB::Exception: Number of arguments for function errorCodeToName doesn't match: passed 3, should be 1: In scope SELECT errorCodeToName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT errorCodeToName([], [], []);) 2025.09.25 21:10:42.703596 [ 2707 ] {ce856fc1-d87f-4349-bd34-5a38165b0718} TCPHandler: Code: 42. DB::Exception: Number of arguments for function errorCodeToName doesn't match: passed 3, should be 1: In scope SELECT errorCodeToName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:42.787715 [ 2707 ] {6f9b8877-45a7-4f0b-b083-ad819f2f67f0} executeQuery: Code: 43. DB::Exception: Argument for function evalMLMethod must have type AggregateFunction - state of aggregate function.: In scope SELECT evalMLMethod([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT evalMLMethod([], [], []);) 2025.09.25 21:10:42.796789 [ 2707 ] {6f9b8877-45a7-4f0b-b083-ad819f2f67f0} TCPHandler: Code: 43. DB::Exception: Argument for function evalMLMethod must have type AggregateFunction - state of aggregate function.: In scope SELECT evalMLMethod([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:42.839379 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:42.875413 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:42.895523 [ 2707 ] {e5c576be-f9c8-4e18-b56e-4e27c257c786} executeQuery: Code: 42. DB::Exception: Number of arguments for function exp doesn't match: passed 3, should be 1: In scope SELECT exp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT exp([], [], []);) 2025.09.25 21:10:42.901172 [ 2707 ] {e5c576be-f9c8-4e18-b56e-4e27c257c786} TCPHandler: Code: 42. DB::Exception: Number of arguments for function exp doesn't match: passed 3, should be 1: In scope SELECT exp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:43.011870 [ 2707 ] {845d5ec7-6669-46a0-af9f-03bc6c0da76f} executeQuery: Code: 42. DB::Exception: Number of arguments for function exp10 doesn't match: passed 3, should be 1: In scope SELECT exp10([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT exp10([], [], []);) 2025.09.25 21:10:43.020245 [ 2707 ] {845d5ec7-6669-46a0-af9f-03bc6c0da76f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function exp10 doesn't match: passed 3, should be 1: In scope SELECT exp10([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:43.098933 [ 2707 ] {3d7b3c15-f7e9-4830-80c5-f71c116aab66} executeQuery: Code: 42. DB::Exception: Number of arguments for function exp2 doesn't match: passed 3, should be 1: In scope SELECT exp2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT exp2([], [], []);) 2025.09.25 21:10:43.104988 [ 2707 ] {3d7b3c15-f7e9-4830-80c5-f71c116aab66} TCPHandler: Code: 42. DB::Exception: Number of arguments for function exp2 doesn't match: passed 3, should be 1: In scope SELECT exp2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:43.162888 [ 2707 ] {93355660-a18b-4f2f-a707-dddfeaf976a3} executeQuery: Code: 42. DB::Exception: Aggregate function exponentialMovingAverage requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT exponentialMovingAverage([], [], []);) 2025.09.25 21:10:43.170523 [ 2707 ] {93355660-a18b-4f2f-a707-dddfeaf976a3} TCPHandler: Code: 42. DB::Exception: Aggregate function exponentialMovingAverage requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:43.364455 [ 2707 ] {08a629b6-b9e1-494b-9a58-ea25c5682f75} executeQuery: Code: 42. DB::Exception: Function exponentialTimeDecayedAvg takes exactly one parameter. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT exponentialTimeDecayedAvg([], [], []);) 2025.09.25 21:10:43.371601 [ 2707 ] {08a629b6-b9e1-494b-9a58-ea25c5682f75} TCPHandler: Code: 42. DB::Exception: Function exponentialTimeDecayedAvg takes exactly one parameter. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:43.452162 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:43.480341 [ 2707 ] {c99ac296-6d32-4741-a2b6-cf3e4f94d0e8} executeQuery: Code: 42. DB::Exception: Function exponentialTimeDecayedCount takes exactly one parameter. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT exponentialTimeDecayedCount([], [], []);) 2025.09.25 21:10:43.488292 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:43.494465 [ 2707 ] {c99ac296-6d32-4741-a2b6-cf3e4f94d0e8} TCPHandler: Code: 42. DB::Exception: Function exponentialTimeDecayedCount takes exactly one parameter. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:43.565878 [ 2707 ] {b478c4f6-fe77-4508-89fe-41185848cb53} executeQuery: Code: 42. DB::Exception: Function exponentialTimeDecayedMax takes exactly one parameter. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT exponentialTimeDecayedMax([], [], []);) 2025.09.25 21:10:43.580383 [ 2707 ] {b478c4f6-fe77-4508-89fe-41185848cb53} TCPHandler: Code: 42. DB::Exception: Function exponentialTimeDecayedMax takes exactly one parameter. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:43.658991 [ 2707 ] {bcc3aba5-8416-45a4-8158-6150f9d54dc0} executeQuery: Code: 42. DB::Exception: Function exponentialTimeDecayedSum takes exactly one parameter. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT exponentialTimeDecayedSum([], [], []);) 2025.09.25 21:10:43.663859 [ 2707 ] {bcc3aba5-8416-45a4-8158-6150f9d54dc0} TCPHandler: Code: 42. DB::Exception: Function exponentialTimeDecayedSum takes exactly one parameter. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:43.719182 [ 2707 ] {861e17d6-09c2-4d0c-93a2-125ec632aa07} executeQuery: Code: 42. DB::Exception: Number of arguments for function extract doesn't match: passed 3, should be 2: In scope SELECT extract([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT extract([], [], []);) 2025.09.25 21:10:43.725858 [ 2707 ] {861e17d6-09c2-4d0c-93a2-125ec632aa07} TCPHandler: Code: 42. DB::Exception: Number of arguments for function extract doesn't match: passed 3, should be 2: In scope SELECT extract([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:43.790594 [ 2707 ] {0e9aefbf-2eff-4f56-906e-39a522048462} executeQuery: Code: 42. DB::Exception: Number of arguments for function extractAll doesn't match: passed 3, should be 2: In scope SELECT extractAll([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT extractAll([], [], []);) 2025.09.25 21:10:43.796670 [ 2707 ] {0e9aefbf-2eff-4f56-906e-39a522048462} TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractAll doesn't match: passed 3, should be 2: In scope SELECT extractAll([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:43.961133 [ 2707 ] {526b4bba-8fe9-456b-b60e-79fc57c64843} executeQuery: Code: 42. DB::Exception: Number of arguments for function extractAllGroupsVertical doesn't match: passed 3, should be 2: In scope SELECT extractAllGroups([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT extractAllGroups([], [], []);) 2025.09.25 21:10:43.972035 [ 2707 ] {526b4bba-8fe9-456b-b60e-79fc57c64843} TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractAllGroupsVertical doesn't match: passed 3, should be 2: In scope SELECT extractAllGroups([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:44.094981 [ 2707 ] {ce06a965-1e1d-490d-ab3a-eb50e6ec7f16} executeQuery: Code: 42. DB::Exception: Number of arguments for function extractAllGroupsHorizontal doesn't match: passed 3, should be 2: In scope SELECT extractAllGroupsHorizontal([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT extractAllGroupsHorizontal([], [], []);) 2025.09.25 21:10:44.100058 [ 2707 ] {ce06a965-1e1d-490d-ab3a-eb50e6ec7f16} TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractAllGroupsHorizontal doesn't match: passed 3, should be 2: In scope SELECT extractAllGroupsHorizontal([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:44.166698 [ 2707 ] {c9b37766-8683-46fb-b824-a35c6d30d5c0} executeQuery: Code: 42. DB::Exception: Number of arguments for function extractAllGroupsVertical doesn't match: passed 3, should be 2: In scope SELECT extractAllGroupsVertical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT extractAllGroupsVertical([], [], []);) 2025.09.25 21:10:44.172191 [ 2707 ] {c9b37766-8683-46fb-b824-a35c6d30d5c0} TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractAllGroupsVertical doesn't match: passed 3, should be 2: In scope SELECT extractAllGroupsVertical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:44.302683 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:44.315034 [ 2707 ] {ea314676-bf04-44bc-b4d1-5d429b5b4c21} executeQuery: Code: 42. DB::Exception: Number of arguments for function extractGroups doesn't match: passed 3, should be 2: In scope SELECT extractGroups([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT extractGroups([], [], []);) 2025.09.25 21:10:44.319213 [ 2707 ] {ea314676-bf04-44bc-b4d1-5d429b5b4c21} TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractGroups doesn't match: passed 3, should be 2: In scope SELECT extractGroups([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:44.324858 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:44.385434 [ 2707 ] {6992f27a-83dc-47ad-bbb0-88c54be15410} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument data_column. Must be String.: In scope SELECT extractKeyValuePairs([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT extractKeyValuePairs([], [], []);) 2025.09.25 21:10:44.392217 [ 2707 ] {6992f27a-83dc-47ad-bbb0-88c54be15410} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument data_column. Must be String.: In scope SELECT extractKeyValuePairs([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:44.508014 [ 2707 ] {03bd69f1-654b-4f90-ba04-24d69a629b57} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument data_column. Must be String.: In scope SELECT extractKeyValuePairsWithEscaping([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT extractKeyValuePairsWithEscaping([], [], []);) 2025.09.25 21:10:44.520287 [ 2707 ] {03bd69f1-654b-4f90-ba04-24d69a629b57} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument data_column. Must be String.: In scope SELECT extractKeyValuePairsWithEscaping([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:44.618184 [ 2707 ] {8248786f-8aa3-4a0f-abf2-b0c6a8ae8ddf} executeQuery: Code: 42. DB::Exception: Number of arguments for function extractTextFromHTML doesn't match: passed 3, should be 1: In scope SELECT extractTextFromHTML([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT extractTextFromHTML([], [], []);) 2025.09.25 21:10:44.632063 [ 2707 ] {8248786f-8aa3-4a0f-abf2-b0c6a8ae8ddf} TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractTextFromHTML doesn't match: passed 3, should be 1: In scope SELECT extractTextFromHTML([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:44.862447 [ 2707 ] {5bb32ca5-cbad-4a13-8389-8e1a36722cd9} executeQuery: Code: 42. DB::Exception: Number of arguments for function extractURLParameter doesn't match: passed 3, should be 2: In scope SELECT extractURLParameter([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT extractURLParameter([], [], []);) 2025.09.25 21:10:44.874601 [ 2707 ] {5bb32ca5-cbad-4a13-8389-8e1a36722cd9} TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractURLParameter doesn't match: passed 3, should be 2: In scope SELECT extractURLParameter([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:45.018470 [ 2707 ] {6d42e0db-02f5-48b2-9af0-e53ca1e2d36e} executeQuery: Code: 42. DB::Exception: Number of arguments for function extractURLParameterNames doesn't match: passed 3, should be 1: In scope SELECT extractURLParameterNames([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT extractURLParameterNames([], [], []);) 2025.09.25 21:10:45.022697 [ 2707 ] {6d42e0db-02f5-48b2-9af0-e53ca1e2d36e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractURLParameterNames doesn't match: passed 3, should be 1: In scope SELECT extractURLParameterNames([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:45.087772 [ 2707 ] {838718d3-890b-4c6c-9b4d-00b8d3a2a384} executeQuery: Code: 42. DB::Exception: Number of arguments for function extractURLParameters doesn't match: passed 3, should be 1: In scope SELECT extractURLParameters([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT extractURLParameters([], [], []);) 2025.09.25 21:10:45.091408 [ 2707 ] {838718d3-890b-4c6c-9b4d-00b8d3a2a384} TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractURLParameters doesn't match: passed 3, should be 1: In scope SELECT extractURLParameters([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:45.122699 [ 2707 ] {8e3b6102-5a62-4c29-b2f6-0b64cca76072} executeQuery: Code: 42. DB::Exception: Number of arguments for function factorial doesn't match: passed 3, should be 1: In scope SELECT factorial([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT factorial([], [], []);) 2025.09.25 21:10:45.125418 [ 2707 ] {8e3b6102-5a62-4c29-b2f6-0b64cca76072} TCPHandler: Code: 42. DB::Exception: Number of arguments for function factorial doesn't match: passed 3, should be 1: In scope SELECT factorial([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:45.445772 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:45.464681 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:45.469497 [ 2707 ] {b9385222-8d14-48c9-8449-0c4d037a4a00} executeQuery: Code: 42. DB::Exception: Number of arguments for function file doesn't match: passed 3, should be 1 or 2: In scope SELECT file([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT file([], [], []);) 2025.09.25 21:10:45.473501 [ 2707 ] {b9385222-8d14-48c9-8449-0c4d037a4a00} TCPHandler: Code: 42. DB::Exception: Number of arguments for function file doesn't match: passed 3, should be 1 or 2: In scope SELECT file([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:45.531084 [ 2707 ] {9996835e-98c5-4615-a864-89a032a0036e} executeQuery: Code: 42. DB::Exception: Arguments size of function filesystemAvailable should be 0 or 1: In scope SELECT filesystemAvailable([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT filesystemAvailable([], [], []);) 2025.09.25 21:10:45.536841 [ 2707 ] {9996835e-98c5-4615-a864-89a032a0036e} TCPHandler: Code: 42. DB::Exception: Arguments size of function filesystemAvailable should be 0 or 1: In scope SELECT filesystemAvailable([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:45.660703 [ 2707 ] {6d568dd5-8c7a-4437-b069-b8f9c3b840ae} executeQuery: Code: 42. DB::Exception: Arguments size of function filesystemCapacity should be 0 or 1: In scope SELECT filesystemCapacity([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT filesystemCapacity([], [], []);) 2025.09.25 21:10:45.665695 [ 2707 ] {6d568dd5-8c7a-4437-b069-b8f9c3b840ae} TCPHandler: Code: 42. DB::Exception: Arguments size of function filesystemCapacity should be 0 or 1: In scope SELECT filesystemCapacity([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:45.730935 [ 2707 ] {dc9ee724-3370-463e-a68c-10e30b6279a5} executeQuery: Code: 42. DB::Exception: Arguments size of function filesystemUnreserved should be 0 or 1: In scope SELECT filesystemUnreserved([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT filesystemUnreserved([], [], []);) 2025.09.25 21:10:45.736487 [ 2707 ] {dc9ee724-3370-463e-a68c-10e30b6279a5} TCPHandler: Code: 42. DB::Exception: Arguments size of function filesystemUnreserved should be 0 or 1: In scope SELECT filesystemUnreserved([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:45.827419 [ 2707 ] {a5d70c29-2b09-429c-bf50-b7fcf0456755} executeQuery: Code: 42. DB::Exception: Number of arguments for function finalizeAggregation doesn't match: passed 3, should be 1: In scope SELECT finalizeAggregation([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT finalizeAggregation([], [], []);) 2025.09.25 21:10:45.833818 [ 2707 ] {a5d70c29-2b09-429c-bf50-b7fcf0456755} TCPHandler: Code: 42. DB::Exception: Number of arguments for function finalizeAggregation doesn't match: passed 3, should be 1: In scope SELECT finalizeAggregation([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:46.014597 [ 2707 ] {f0e717d9-9c74-4823-b99f-f3fc3e33611b} executeQuery: Code: 42. DB::Exception: Number of arguments for function firstLine doesn't match: passed 3, should be 1: In scope SELECT firstLine([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT firstLine([], [], []);) 2025.09.25 21:10:46.020223 [ 2707 ] {f0e717d9-9c74-4823-b99f-f3fc3e33611b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function firstLine doesn't match: passed 3, should be 1: In scope SELECT firstLine([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:46.087993 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:46.105392 [ 2707 ] {16784b30-9542-4d01-a294-136afb800517} executeQuery: Code: 42. DB::Exception: Number of arguments for function firstSignificantSubdomain doesn't match: passed 3, should be 1: In scope SELECT firstSignificantSubdomain([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT firstSignificantSubdomain([], [], []);) 2025.09.25 21:10:46.111629 [ 2707 ] {16784b30-9542-4d01-a294-136afb800517} TCPHandler: Code: 42. DB::Exception: Number of arguments for function firstSignificantSubdomain doesn't match: passed 3, should be 1: In scope SELECT firstSignificantSubdomain([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:46.123399 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:46.265151 [ 2707 ] {13b08dec-2d19-473d-b993-e8581f01d7b4} executeQuery: Code: 42. DB::Exception: Number of arguments for function firstSignificantSubdomainCustom doesn't match: passed 3, should be 2: In scope SELECT firstSignificantSubdomainCustom([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT firstSignificantSubdomainCustom([], [], []);) 2025.09.25 21:10:46.274412 [ 2707 ] {13b08dec-2d19-473d-b993-e8581f01d7b4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function firstSignificantSubdomainCustom doesn't match: passed 3, should be 2: In scope SELECT firstSignificantSubdomainCustom([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:46.338522 [ 2707 ] {0dd001fe-a726-43ab-af17-219d3abc26df} executeQuery: Code: 42. DB::Exception: Number of arguments for function firstSignificantSubdomainCustomRFC doesn't match: passed 3, should be 2: In scope SELECT firstSignificantSubdomainCustomRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT firstSignificantSubdomainCustomRFC([], [], []);) 2025.09.25 21:10:46.345833 [ 2707 ] {0dd001fe-a726-43ab-af17-219d3abc26df} TCPHandler: Code: 42. DB::Exception: Number of arguments for function firstSignificantSubdomainCustomRFC doesn't match: passed 3, should be 2: In scope SELECT firstSignificantSubdomainCustomRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:46.375809 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:46.418974 [ 2707 ] {34c77644-04e5-4a35-9d89-8eb9c9b7a074} executeQuery: Code: 42. DB::Exception: Number of arguments for function firstSignificantSubdomainRFC doesn't match: passed 3, should be 1: In scope SELECT firstSignificantSubdomainRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT firstSignificantSubdomainRFC([], [], []);) 2025.09.25 21:10:46.425384 [ 2707 ] {34c77644-04e5-4a35-9d89-8eb9c9b7a074} TCPHandler: Code: 42. DB::Exception: Number of arguments for function firstSignificantSubdomainRFC doesn't match: passed 3, should be 1: In scope SELECT firstSignificantSubdomainRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:46.450773 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:46.488851 [ 2707 ] {7866f58c-c004-42d9-ae03-d8be2d1e0f56} executeQuery: Code: 42. DB::Exception: Aggregate function any requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT first_value([], [], []);) 2025.09.25 21:10:46.496710 [ 2707 ] {7866f58c-c004-42d9-ae03-d8be2d1e0f56} TCPHandler: Code: 42. DB::Exception: Aggregate function any requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:46.647569 [ 2707 ] {7439486c-a093-44df-94c1-5e5751a56903} executeQuery: Code: 42. DB::Exception: Aggregate function any_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT first_value_respect_nulls([], [], []);) 2025.09.25 21:10:46.651699 [ 2707 ] {7439486c-a093-44df-94c1-5e5751a56903} TCPHandler: Code: 42. DB::Exception: Aggregate function any_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:46.707631 [ 2707 ] {83f0e597-6cdc-44bc-b849-060275a4f5ff} executeQuery: Code: 43. DB::Exception: First argument (trace) for function flameGraph must be Array(UInt64), but it has type Array(Nothing). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT flameGraph([], [], []);) 2025.09.25 21:10:46.713500 [ 2707 ] {83f0e597-6cdc-44bc-b849-060275a4f5ff} TCPHandler: Code: 43. DB::Exception: First argument (trace) for function flameGraph must be Array(UInt64), but it has type Array(Nothing). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:46.901277 [ 2707 ] {d6e6c4f1-7ce5-434d-9bd2-ef1a47442d6f} executeQuery: Code: 42. DB::Exception: Number of arguments for function arrayFlatten doesn't match: passed 3, should be 1: In scope SELECT flatten([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT flatten([], [], []);) 2025.09.25 21:10:46.908959 [ 2707 ] {d6e6c4f1-7ce5-434d-9bd2-ef1a47442d6f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function arrayFlatten doesn't match: passed 3, should be 1: In scope SELECT flatten([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:46.961553 [ 2707 ] {1c939430-2138-4149-b7d6-ee500aba8ac8} executeQuery: Code: 42. DB::Exception: Number of arguments for function flattenTuple doesn't match: passed 3, should be 1: In scope SELECT flattenTuple([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT flattenTuple([], [], []);) 2025.09.25 21:10:46.968109 [ 2707 ] {1c939430-2138-4149-b7d6-ee500aba8ac8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function flattenTuple doesn't match: passed 3, should be 1: In scope SELECT flattenTuple([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:46.980780 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:46.997495 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:47.026089 [ 2707 ] {b92f271d-ad6e-4ed8-967b-b3e84e004a3a} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'floor'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT floor([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT floor([], [], []);) 2025.09.25 21:10:47.031344 [ 2707 ] {b92f271d-ad6e-4ed8-967b-b3e84e004a3a} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'floor'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT floor([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:47.090173 [ 2707 ] {398f60bf-2c4f-4ebf-ab6c-3b442313de10} executeQuery: Code: 170. DB::Exception: Bad get: has Array, requested String: In scope SELECT format([], [], []). (BAD_GET) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT format([], [], []);) 2025.09.25 21:10:47.099166 [ 2707 ] {398f60bf-2c4f-4ebf-ab6c-3b442313de10} TCPHandler: Code: 170. DB::Exception: Bad get: has Array, requested String: In scope SELECT format([], [], []). (BAD_GET) 2025.09.25 21:10:47.226684 [ 2707 ] {7d8c4022-b2d3-4eff-bafb-5915c04b6733} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function formatDateTime. Should be a date or a date with time: In scope SELECT formatDateTime([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT formatDateTime([], [], []);) 2025.09.25 21:10:47.228913 [ 2707 ] {7d8c4022-b2d3-4eff-bafb-5915c04b6733} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function formatDateTime. Should be a date or a date with time: In scope SELECT formatDateTime([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:47.334055 [ 2707 ] {60bb9410-e42d-4acb-8452-ab1e8921ac00} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function formatDateTimeInJodaSyntax. Should be a date or a date with time: In scope SELECT formatDateTimeInJodaSyntax([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT formatDateTimeInJodaSyntax([], [], []);) 2025.09.25 21:10:47.342455 [ 2707 ] {60bb9410-e42d-4acb-8452-ab1e8921ac00} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function formatDateTimeInJodaSyntax. Should be a date or a date with time: In scope SELECT formatDateTimeInJodaSyntax([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:47.405423 [ 2707 ] {5c15795d-3e18-44cc-975b-ceac04e9463d} executeQuery: Code: 42. DB::Exception: Number of arguments for function formatQuery doesn't match: passed 3, should be 1: In scope SELECT formatQuery([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT formatQuery([], [], []);) 2025.09.25 21:10:47.460431 [ 2707 ] {5c15795d-3e18-44cc-975b-ceac04e9463d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function formatQuery doesn't match: passed 3, should be 1: In scope SELECT formatQuery([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:47.513911 [ 2707 ] {770b4b56-2243-47e8-977b-b27cfd973fa8} executeQuery: Code: 42. DB::Exception: Number of arguments for function formatQueryOrNull doesn't match: passed 3, should be 1: In scope SELECT formatQueryOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT formatQueryOrNull([], [], []);) 2025.09.25 21:10:47.520844 [ 2707 ] {770b4b56-2243-47e8-977b-b27cfd973fa8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function formatQueryOrNull doesn't match: passed 3, should be 1: In scope SELECT formatQueryOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:47.599685 [ 2707 ] {bb8d4474-a5f6-493e-88aa-247220867abc} executeQuery: Code: 42. DB::Exception: Number of arguments for function formatQuerySingleLine doesn't match: passed 3, should be 1: In scope SELECT formatQuerySingleLine([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT formatQuerySingleLine([], [], []);) 2025.09.25 21:10:47.605426 [ 2707 ] {bb8d4474-a5f6-493e-88aa-247220867abc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function formatQuerySingleLine doesn't match: passed 3, should be 1: In scope SELECT formatQuerySingleLine([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:47.618328 [ 7168 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:47.665911 [ 2707 ] {e85154dc-910f-43d1-88e6-d1bebfe2220a} executeQuery: Code: 42. DB::Exception: Number of arguments for function formatQuerySingleLineOrNull doesn't match: passed 3, should be 1: In scope SELECT formatQuerySingleLineOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT formatQuerySingleLineOrNull([], [], []);) 2025.09.25 21:10:47.674194 [ 2707 ] {e85154dc-910f-43d1-88e6-d1bebfe2220a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function formatQuerySingleLineOrNull doesn't match: passed 3, should be 1: In scope SELECT formatQuerySingleLineOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:47.675836 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:47.731626 [ 2707 ] {12d23b7c-5836-48bf-815d-a0e0eccfec5f} executeQuery: Code: 42. DB::Exception: Number of arguments for function formatReadableDecimalSize doesn't match: passed 3, should be 1: In scope SELECT formatReadableDecimalSize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT formatReadableDecimalSize([], [], []);) 2025.09.25 21:10:47.736664 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:47.795801 [ 2707 ] {12d23b7c-5836-48bf-815d-a0e0eccfec5f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function formatReadableDecimalSize doesn't match: passed 3, should be 1: In scope SELECT formatReadableDecimalSize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:47.848806 [ 2707 ] {747ffd9e-c794-4bce-93e6-1ea84c7aa167} executeQuery: Code: 42. DB::Exception: Number of arguments for function formatReadableQuantity doesn't match: passed 3, should be 1: In scope SELECT formatReadableQuantity([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT formatReadableQuantity([], [], []);) 2025.09.25 21:10:47.858694 [ 2707 ] {747ffd9e-c794-4bce-93e6-1ea84c7aa167} TCPHandler: Code: 42. DB::Exception: Number of arguments for function formatReadableQuantity doesn't match: passed 3, should be 1: In scope SELECT formatReadableQuantity([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:47.915084 [ 2707 ] {7f1e3c71-0346-4890-ab15-dd2fba361b62} executeQuery: Code: 42. DB::Exception: Number of arguments for function formatReadableSize doesn't match: passed 3, should be 1: In scope SELECT formatReadableSize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT formatReadableSize([], [], []);) 2025.09.25 21:10:47.920502 [ 2707 ] {7f1e3c71-0346-4890-ab15-dd2fba361b62} TCPHandler: Code: 42. DB::Exception: Number of arguments for function formatReadableSize doesn't match: passed 3, should be 1: In scope SELECT formatReadableSize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:47.981295 [ 2707 ] {12fe97fe-cd9c-44a7-8a83-92f59e37b8e9} executeQuery: Code: 43. DB::Exception: Cannot format Array(Nothing) as time delta: In scope SELECT formatReadableTimeDelta([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT formatReadableTimeDelta([], [], []);) 2025.09.25 21:10:47.986835 [ 2707 ] {12fe97fe-cd9c-44a7-8a83-92f59e37b8e9} TCPHandler: Code: 43. DB::Exception: Cannot format Array(Nothing) as time delta: In scope SELECT formatReadableTimeDelta([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:48.040451 [ 2707 ] {74b2c553-df76-4ad6-a5e4-96ba312bf164} executeQuery: Code: 43. DB::Exception: First argument to formatRow must be a format name: In scope SELECT formatRow([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT formatRow([], [], []);) 2025.09.25 21:10:48.047433 [ 2707 ] {74b2c553-df76-4ad6-a5e4-96ba312bf164} TCPHandler: Code: 43. DB::Exception: First argument to formatRow must be a format name: In scope SELECT formatRow([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:48.107086 [ 2707 ] {4c121638-5aed-4d25-839a-0854c1702247} executeQuery: Code: 43. DB::Exception: First argument to formatRowNoNewline must be a format name: In scope SELECT formatRowNoNewline([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT formatRowNoNewline([], [], []);) 2025.09.25 21:10:48.111953 [ 2707 ] {4c121638-5aed-4d25-839a-0854c1702247} TCPHandler: Code: 43. DB::Exception: First argument to formatRowNoNewline must be a format name: In scope SELECT formatRowNoNewline([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:48.233231 [ 2707 ] {ff05859c-a6e0-4cf6-9960-6cd4d9dbd88e} executeQuery: Code: 42. DB::Exception: Number of arguments for function fragment doesn't match: passed 3, should be 1: In scope SELECT fragment([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT fragment([], [], []);) 2025.09.25 21:10:48.237955 [ 2707 ] {ff05859c-a6e0-4cf6-9960-6cd4d9dbd88e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function fragment doesn't match: passed 3, should be 1: In scope SELECT fragment([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:48.297808 [ 2707 ] {09b7a36f-ef06-46ac-a420-7803e31908a1} executeQuery: Code: 42. DB::Exception: Number of arguments for function fromDaysSinceYearZero doesn't match: passed 3, should be 1: In scope SELECT fromDaysSinceYearZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT fromDaysSinceYearZero([], [], []);) 2025.09.25 21:10:48.303326 [ 2707 ] {09b7a36f-ef06-46ac-a420-7803e31908a1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function fromDaysSinceYearZero doesn't match: passed 3, should be 1: In scope SELECT fromDaysSinceYearZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:48.353417 [ 2707 ] {feb3586a-5c26-481d-b4c7-99a294c2d9bf} executeQuery: Code: 42. DB::Exception: Number of arguments for function fromDaysSinceYearZero32 doesn't match: passed 3, should be 1: In scope SELECT fromDaysSinceYearZero32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT fromDaysSinceYearZero32([], [], []);) 2025.09.25 21:10:48.458087 [ 2707 ] {feb3586a-5c26-481d-b4c7-99a294c2d9bf} TCPHandler: Code: 42. DB::Exception: Number of arguments for function fromDaysSinceYearZero32 doesn't match: passed 3, should be 1: In scope SELECT fromDaysSinceYearZero32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:48.505784 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:48.521969 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:48.524212 [ 2707 ] {a09f66c9-d850-4f44-8ef8-bf230b6a0902} executeQuery: Code: 42. DB::Exception: Number of arguments for function fromModifiedJulianDay doesn't match: passed 3, should be 1: In scope SELECT fromModifiedJulianDay([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT fromModifiedJulianDay([], [], []);) 2025.09.25 21:10:48.530463 [ 2707 ] {a09f66c9-d850-4f44-8ef8-bf230b6a0902} TCPHandler: Code: 42. DB::Exception: Number of arguments for function fromModifiedJulianDay doesn't match: passed 3, should be 1: In scope SELECT fromModifiedJulianDay([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:48.765429 [ 2707 ] {72e916b7-8f7c-4f31-8aef-0cca3bab13cf} executeQuery: Code: 42. DB::Exception: Number of arguments for function fromModifiedJulianDayOrNull doesn't match: passed 3, should be 1: In scope SELECT fromModifiedJulianDayOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT fromModifiedJulianDayOrNull([], [], []);) 2025.09.25 21:10:48.770446 [ 2707 ] {72e916b7-8f7c-4f31-8aef-0cca3bab13cf} TCPHandler: Code: 42. DB::Exception: Number of arguments for function fromModifiedJulianDayOrNull doesn't match: passed 3, should be 1: In scope SELECT fromModifiedJulianDayOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:48.824183 [ 2707 ] {0009d965-9fba-49b9-bf0d-e2bf761ffa78} executeQuery: Code: 42. DB::Exception: Number of arguments for function fromUTCTimestamp doesn't match: passed 3, should be 2: In scope SELECT fromUTCTimestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT fromUTCTimestamp([], [], []);) 2025.09.25 21:10:48.829091 [ 2707 ] {0009d965-9fba-49b9-bf0d-e2bf761ffa78} TCPHandler: Code: 42. DB::Exception: Number of arguments for function fromUTCTimestamp doesn't match: passed 3, should be 2: In scope SELECT fromUTCTimestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:48.949733 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:48.961974 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:48.986659 [ 2707 ] {0d30ebdb-27fa-4667-9194-01f4bbccbf9a} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function fromUnixTimestamp when arguments size is 2 or 3. Should be a integer or a date with time: In scope SELECT fromUnixTimestamp([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT fromUnixTimestamp([], [], []);) 2025.09.25 21:10:48.988967 [ 2707 ] {0d30ebdb-27fa-4667-9194-01f4bbccbf9a} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function fromUnixTimestamp when arguments size is 2 or 3. Should be a integer or a date with time: In scope SELECT fromUnixTimestamp([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:49.039768 [ 2707 ] {e6d55418-9257-4dc0-af67-24617546d71d} executeQuery: Code: 42. DB::Exception: Function fromUnixTimestamp64Micro takes one or two arguments: In scope SELECT fromUnixTimestamp64Micro([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT fromUnixTimestamp64Micro([], [], []);) 2025.09.25 21:10:49.044903 [ 2707 ] {e6d55418-9257-4dc0-af67-24617546d71d} TCPHandler: Code: 42. DB::Exception: Function fromUnixTimestamp64Micro takes one or two arguments: In scope SELECT fromUnixTimestamp64Micro([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:49.109626 [ 2707 ] {379ff99c-f96d-47c8-a375-87d6ad3334f6} executeQuery: Code: 42. DB::Exception: Function fromUnixTimestamp64Milli takes one or two arguments: In scope SELECT fromUnixTimestamp64Milli([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT fromUnixTimestamp64Milli([], [], []);) 2025.09.25 21:10:49.120022 [ 2707 ] {379ff99c-f96d-47c8-a375-87d6ad3334f6} TCPHandler: Code: 42. DB::Exception: Function fromUnixTimestamp64Milli takes one or two arguments: In scope SELECT fromUnixTimestamp64Milli([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:49.194402 [ 2707 ] {aea3cedd-1cb8-4c51-969b-535d5a3df84a} executeQuery: Code: 42. DB::Exception: Function fromUnixTimestamp64Nano takes one or two arguments: In scope SELECT fromUnixTimestamp64Nano([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT fromUnixTimestamp64Nano([], [], []);) 2025.09.25 21:10:49.201278 [ 2707 ] {aea3cedd-1cb8-4c51-969b-535d5a3df84a} TCPHandler: Code: 42. DB::Exception: Function fromUnixTimestamp64Nano takes one or two arguments: In scope SELECT fromUnixTimestamp64Nano([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:49.260932 [ 2707 ] {105f8a2e-cfea-4cab-bc0f-fb0aaf1045a1} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function fromUnixTimestampInJodaSyntax when arguments size is 2 or 3. Should be a integer or a date with time: In scope SELECT fromUnixTimestampInJodaSyntax([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT fromUnixTimestampInJodaSyntax([], [], []);) 2025.09.25 21:10:49.266058 [ 2707 ] {105f8a2e-cfea-4cab-bc0f-fb0aaf1045a1} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function fromUnixTimestampInJodaSyntax when arguments size is 2 or 3. Should be a integer or a date with time: In scope SELECT fromUnixTimestampInJodaSyntax([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:49.332187 [ 2707 ] {e20bd547-cae7-42c8-8972-d8f8e4953c23} executeQuery: Code: 42. DB::Exception: Number of arguments for function fromUTCTimestamp doesn't match: passed 3, should be 2: In scope SELECT from_utc_timestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT from_utc_timestamp([], [], []);) 2025.09.25 21:10:49.336899 [ 2707 ] {e20bd547-cae7-42c8-8972-d8f8e4953c23} TCPHandler: Code: 42. DB::Exception: Number of arguments for function fromUTCTimestamp doesn't match: passed 3, should be 2: In scope SELECT from_utc_timestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:49.420046 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:49.428927 [ 2707 ] {d20b4f98-a514-424b-bf86-b072b32d6f82} executeQuery: Code: 42. DB::Exception: Number of arguments for function FQDN doesn't match: passed 3, should be 0: In scope SELECT fullHostName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT fullHostName([], [], []);) 2025.09.25 21:10:49.441149 [ 2707 ] {d20b4f98-a514-424b-bf86-b072b32d6f82} TCPHandler: Code: 42. DB::Exception: Number of arguments for function FQDN doesn't match: passed 3, should be 0: In scope SELECT fullHostName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:49.467526 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:49.547076 [ 2707 ] {99ddb04a-3eca-4e07-bb9b-2d695eb4c2ae} executeQuery: Code: 42. DB::Exception: Number of arguments for function fuzzBits doesn't match: passed 3, should be 2: In scope SELECT fuzzBits([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT fuzzBits([], [], []);) 2025.09.25 21:10:49.554218 [ 2707 ] {99ddb04a-3eca-4e07-bb9b-2d695eb4c2ae} TCPHandler: Code: 42. DB::Exception: Number of arguments for function fuzzBits doesn't match: passed 3, should be 2: In scope SELECT fuzzBits([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:49.825089 [ 2707 ] {7c425141-1cd0-4cf6-85ae-1603d8f67c67} executeQuery: Code: 42. DB::Exception: Number of arguments for function gcd doesn't match: passed 3, should be 2: In scope SELECT gcd([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT gcd([], [], []);) 2025.09.25 21:10:49.834101 [ 2707 ] {7c425141-1cd0-4cf6-85ae-1603d8f67c67} TCPHandler: Code: 42. DB::Exception: Number of arguments for function gcd doesn't match: passed 3, should be 2: In scope SELECT gcd([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:49.906879 [ 2707 ] {3be09d0d-43e4-4ad5-9b69-5ed39b989ba0} executeQuery: Code: 42. DB::Exception: Number of arguments for function generateRandomStructure doesn't match: passed 3, expected from 0 to 2: In scope SELECT generateRandomStructure([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT generateRandomStructure([], [], []);) 2025.09.25 21:10:49.911665 [ 2707 ] {3be09d0d-43e4-4ad5-9b69-5ed39b989ba0} TCPHandler: Code: 42. DB::Exception: Number of arguments for function generateRandomStructure doesn't match: passed 3, expected from 0 to 2: In scope SELECT generateRandomStructure([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:50.071198 [ 2707 ] {2ef64cc4-6bf5-4697-950f-010c6f802805} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'generateSnowflakeID'. Expected 2 optional arguments, got 3 arguments: In scope SELECT generateSnowflakeID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT generateSnowflakeID([], [], []);) 2025.09.25 21:10:50.078551 [ 2707 ] {2ef64cc4-6bf5-4697-950f-010c6f802805} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'generateSnowflakeID'. Expected 2 optional arguments, got 3 arguments: In scope SELECT generateSnowflakeID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:50.185949 [ 2707 ] {81093d59-d6ce-4105-99c7-dcaf8f68ab5e} executeQuery: Code: 34. DB::Exception: Number of arguments for function generateULID doesn't match: passed 3, should be 0 or 1.: In scope SELECT generateULID([], [], []). (TOO_MANY_ARGUMENTS_FOR_FUNCTION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT generateULID([], [], []);) 2025.09.25 21:10:50.191960 [ 2707 ] {81093d59-d6ce-4105-99c7-dcaf8f68ab5e} TCPHandler: Code: 34. DB::Exception: Number of arguments for function generateULID doesn't match: passed 3, should be 0 or 1.: In scope SELECT generateULID([], [], []). (TOO_MANY_ARGUMENTS_FOR_FUNCTION) 2025.09.25 21:10:50.245683 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:50.250861 [ 2707 ] {be994794-48ed-461f-857d-0dbf0de4f4ac} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'generateUUIDv4'. Expected 1 optional argument, got 3 arguments: In scope SELECT generateUUIDv4([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT generateUUIDv4([], [], []);) 2025.09.25 21:10:50.255485 [ 2707 ] {be994794-48ed-461f-857d-0dbf0de4f4ac} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'generateUUIDv4'. Expected 1 optional argument, got 3 arguments: In scope SELECT generateUUIDv4([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:50.272031 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:50.390281 [ 2707 ] {61625b3b-6ce6-4563-af05-7f2849429a7d} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'generateUUIDv7'. Expected 1 optional argument, got 3 arguments: In scope SELECT generateUUIDv7([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT generateUUIDv7([], [], []);) 2025.09.25 21:10:50.469262 [ 2707 ] {61625b3b-6ce6-4563-af05-7f2849429a7d} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'generateUUIDv7'. Expected 1 optional argument, got 3 arguments: In scope SELECT generateUUIDv7([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:50.536173 [ 2707 ] {0f43e5ba-2310-449b-917a-4531a5f63734} executeQuery: Code: 42. DB::Exception: Number of arguments for function geoDistance doesn't match: passed 3, should be 4: In scope SELECT geoDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT geoDistance([], [], []);) 2025.09.25 21:10:50.544512 [ 2707 ] {0f43e5ba-2310-449b-917a-4531a5f63734} TCPHandler: Code: 42. DB::Exception: Number of arguments for function geoDistance doesn't match: passed 3, should be 4: In scope SELECT geoDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:50.584377 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:50.604023 [ 2707 ] {6c82cecf-ae81-4f88-909c-8ed55a3f8a8e} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument 1 of function geoToH3. Must be Float64: In scope SELECT geoToH3([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT geoToH3([], [], []);) 2025.09.25 21:10:50.608731 [ 2707 ] {6c82cecf-ae81-4f88-909c-8ed55a3f8a8e} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument 1 of function geoToH3. Must be Float64: In scope SELECT geoToH3([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:50.619050 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:50.660069 [ 2707 ] {dadc6196-9764-4d72-9a15-7bade8e098f6} executeQuery: Code: 42. DB::Exception: Number of arguments for function geoToS2 doesn't match: passed 3, should be 2: In scope SELECT geoToS2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT geoToS2([], [], []);) 2025.09.25 21:10:50.665736 [ 2707 ] {dadc6196-9764-4d72-9a15-7bade8e098f6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function geoToS2 doesn't match: passed 3, should be 2: In scope SELECT geoToS2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:50.716927 [ 2707 ] {9c3d416d-ca3c-4bfa-a011-e252ff6789b8} executeQuery: Code: 42. DB::Exception: Number of arguments for function geohashDecode doesn't match: passed 3, should be 1: In scope SELECT geohashDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT geohashDecode([], [], []);) 2025.09.25 21:10:50.721837 [ 2707 ] {9c3d416d-ca3c-4bfa-a011-e252ff6789b8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function geohashDecode doesn't match: passed 3, should be 1: In scope SELECT geohashDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:50.780142 [ 2707 ] {f9f76465-fed2-4d92-8419-fa3fcbd7f749} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'longitude' to function 'geohashEncode'. Expected: Float*, got: Array(Nothing): In scope SELECT geohashEncode([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT geohashEncode([], [], []);) 2025.09.25 21:10:50.786729 [ 2707 ] {f9f76465-fed2-4d92-8419-fa3fcbd7f749} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'longitude' to function 'geohashEncode'. Expected: Float*, got: Array(Nothing): In scope SELECT geohashEncode([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:50.847782 [ 2707 ] {6979a3f5-e3cb-4d10-9823-dff935ef23c0} executeQuery: Code: 42. DB::Exception: Number of arguments for function geohashesInBox doesn't match: passed 3, should be 5: In scope SELECT geohashesInBox([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT geohashesInBox([], [], []);) 2025.09.25 21:10:50.850390 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:50.853748 [ 2707 ] {6979a3f5-e3cb-4d10-9823-dff935ef23c0} TCPHandler: Code: 42. DB::Exception: Number of arguments for function geohashesInBox doesn't match: passed 3, should be 5: In scope SELECT geohashesInBox([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:50.879741 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:50.905184 [ 2707 ] {daf30c56-f133-4952-bae1-d5eb89a4aa48} executeQuery: Code: 446. DB::Exception: The function getClientHTTPHeader requires setting `allow_get_client_http_header` to be enabled. (FUNCTION_NOT_ALLOWED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT getClientHTTPHeader([], [], []);) 2025.09.25 21:10:50.909583 [ 2707 ] {daf30c56-f133-4952-bae1-d5eb89a4aa48} TCPHandler: Code: 446. DB::Exception: The function getClientHTTPHeader requires setting `allow_get_client_http_header` to be enabled. (FUNCTION_NOT_ALLOWED) 2025.09.25 21:10:50.962437 [ 2707 ] {7813b977-a475-4402-a6a8-33eb8af5e59b} executeQuery: Code: 42. DB::Exception: Number of arguments for function getMacro doesn't match: passed 3, should be 1: In scope SELECT getMacro([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT getMacro([], [], []);) 2025.09.25 21:10:50.967397 [ 2707 ] {7813b977-a475-4402-a6a8-33eb8af5e59b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function getMacro doesn't match: passed 3, should be 1: In scope SELECT getMacro([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:51.028307 [ 2707 ] {b3f81b1a-06ef-49e1-867d-e0e66d6c79fa} executeQuery: Code: 42. DB::Exception: Number of arguments for function getOSKernelVersion doesn't match: passed 3, should be 0: In scope SELECT getOSKernelVersion([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT getOSKernelVersion([], [], []);) 2025.09.25 21:10:51.048055 [ 2707 ] {b3f81b1a-06ef-49e1-867d-e0e66d6c79fa} TCPHandler: Code: 42. DB::Exception: Number of arguments for function getOSKernelVersion doesn't match: passed 3, should be 0: In scope SELECT getOSKernelVersion([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:51.118629 [ 2707 ] {42a3b5b2-5933-49e6-a33f-7bca7e2df144} executeQuery: Code: 42. DB::Exception: Number of arguments for function getServerPort doesn't match: passed 3, should be 1: In scope SELECT getServerPort([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT getServerPort([], [], []);) 2025.09.25 21:10:51.178352 [ 2707 ] {42a3b5b2-5933-49e6-a33f-7bca7e2df144} TCPHandler: Code: 42. DB::Exception: Number of arguments for function getServerPort doesn't match: passed 3, should be 1: In scope SELECT getServerPort([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:51.240762 [ 2707 ] {b8e9cee6-6531-4492-86ce-58465b687fae} executeQuery: Code: 42. DB::Exception: Number of arguments for function getSetting doesn't match: passed 3, should be 1: In scope SELECT getSetting([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT getSetting([], [], []);) 2025.09.25 21:10:51.247010 [ 2707 ] {b8e9cee6-6531-4492-86ce-58465b687fae} TCPHandler: Code: 42. DB::Exception: Number of arguments for function getSetting doesn't match: passed 3, should be 1: In scope SELECT getSetting([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:51.333161 [ 2707 ] {fcfd2e39-5405-4605-8cb2-06eed6740d23} executeQuery: Code: 42. DB::Exception: Number of arguments for function getSizeOfEnumType doesn't match: passed 3, should be 1: In scope SELECT getSizeOfEnumType([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT getSizeOfEnumType([], [], []);) 2025.09.25 21:10:51.349674 [ 2707 ] {fcfd2e39-5405-4605-8cb2-06eed6740d23} TCPHandler: Code: 42. DB::Exception: Number of arguments for function getSizeOfEnumType doesn't match: passed 3, should be 1: In scope SELECT getSizeOfEnumType([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:51.427342 [ 2707 ] {3a5b34a8-ef5c-4186-9934-dbf2249f0615} executeQuery: Code: 42. DB::Exception: Number of arguments for function getSubcolumn doesn't match: passed 3, should be 2: In scope SELECT getSubcolumn([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT getSubcolumn([], [], []);) 2025.09.25 21:10:51.443698 [ 2707 ] {3a5b34a8-ef5c-4186-9934-dbf2249f0615} TCPHandler: Code: 42. DB::Exception: Number of arguments for function getSubcolumn doesn't match: passed 3, should be 2: In scope SELECT getSubcolumn([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:51.530448 [ 2707 ] {e7343c9f-4e6c-4bab-8048-c866132f437d} executeQuery: Code: 42. DB::Exception: Number of arguments for function getTypeSerializationStreams doesn't match: passed 3, should be 1: In scope SELECT getTypeSerializationStreams([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT getTypeSerializationStreams([], [], []);) 2025.09.25 21:10:51.538065 [ 2707 ] {e7343c9f-4e6c-4bab-8048-c866132f437d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function getTypeSerializationStreams doesn't match: passed 3, should be 1: In scope SELECT getTypeSerializationStreams([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:51.602026 [ 2707 ] {ad113083-ddfb-4338-953d-72130b6bd785} executeQuery: Code: 42. DB::Exception: Function 'globalIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT globalIn([], [], []);) 2025.09.25 21:10:51.607329 [ 2707 ] {ad113083-ddfb-4338-953d-72130b6bd785} TCPHandler: Code: 42. DB::Exception: Function 'globalIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:51.672579 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:51.673720 [ 2707 ] {a459a4a9-ba87-422d-8156-cded12cce363} executeQuery: Code: 42. DB::Exception: Function 'globalInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT globalInIgnoreSet([], [], []);) 2025.09.25 21:10:51.680531 [ 2707 ] {a459a4a9-ba87-422d-8156-cded12cce363} TCPHandler: Code: 42. DB::Exception: Function 'globalInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:51.695274 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:51.739021 [ 2707 ] {8c7c71c2-a0c9-4d8e-9755-292b49040983} executeQuery: Code: 42. DB::Exception: Function 'globalNotIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT globalNotIn([], [], []);) 2025.09.25 21:10:51.746680 [ 2707 ] {8c7c71c2-a0c9-4d8e-9755-292b49040983} TCPHandler: Code: 42. DB::Exception: Function 'globalNotIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:51.853563 [ 2707 ] {6c6d01d3-e11a-416e-b5c8-656a0f4b0612} executeQuery: Code: 42. DB::Exception: Function 'globalNotInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT globalNotInIgnoreSet([], [], []);) 2025.09.25 21:10:51.860163 [ 2707 ] {6c6d01d3-e11a-416e-b5c8-656a0f4b0612} TCPHandler: Code: 42. DB::Exception: Function 'globalNotInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:52.027158 [ 2707 ] {bcde59b8-0456-446d-a463-6ef25ff5e402} executeQuery: Code: 42. DB::Exception: Function 'globalNotNullIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT globalNotNullIn([], [], []);) 2025.09.25 21:10:52.036445 [ 2707 ] {bcde59b8-0456-446d-a463-6ef25ff5e402} TCPHandler: Code: 42. DB::Exception: Function 'globalNotNullIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:52.162273 [ 2707 ] {deb56897-198d-456d-a3bf-0d5cf0c8afb6} executeQuery: Code: 42. DB::Exception: Function 'globalNotNullInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT globalNotNullInIgnoreSet([], [], []);) 2025.09.25 21:10:52.167544 [ 2707 ] {deb56897-198d-456d-a3bf-0d5cf0c8afb6} TCPHandler: Code: 42. DB::Exception: Function 'globalNotNullInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:52.288045 [ 2707 ] {c9b84525-c77f-449d-9d23-8934056a889f} executeQuery: Code: 42. DB::Exception: Function 'globalNullIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT globalNullIn([], [], []);) 2025.09.25 21:10:52.293136 [ 2707 ] {c9b84525-c77f-449d-9d23-8934056a889f} TCPHandler: Code: 42. DB::Exception: Function 'globalNullIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:52.353757 [ 2707 ] {141036dd-a991-4ff1-a4e3-44343e0aeb43} executeQuery: Code: 42. DB::Exception: Function 'globalNullInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT globalNullInIgnoreSet([], [], []);) 2025.09.25 21:10:52.359430 [ 2707 ] {141036dd-a991-4ff1-a4e3-44343e0aeb43} TCPHandler: Code: 42. DB::Exception: Function 'globalNullInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:52.401989 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:52.435834 [ 2707 ] {5f8fc1fa-570e-4761-8eac-0e42bc06644c} executeQuery: Code: 42. DB::Exception: Number of arguments for function globalVariable doesn't match: passed 3, should be 1: In scope SELECT globalVariable([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT globalVariable([], [], []);) 2025.09.25 21:10:52.441651 [ 2707 ] {5f8fc1fa-570e-4761-8eac-0e42bc06644c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function globalVariable doesn't match: passed 3, should be 1: In scope SELECT globalVariable([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:52.456894 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:52.598759 [ 2707 ] {c23a48e3-3882-40d7-b888-044e3f223937} executeQuery: Code: 42. DB::Exception: Number of arguments for function greatCircleAngle doesn't match: passed 3, should be 4: In scope SELECT greatCircleAngle([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT greatCircleAngle([], [], []);) 2025.09.25 21:10:52.605338 [ 2707 ] {c23a48e3-3882-40d7-b888-044e3f223937} TCPHandler: Code: 42. DB::Exception: Number of arguments for function greatCircleAngle doesn't match: passed 3, should be 4: In scope SELECT greatCircleAngle([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:52.669178 [ 2707 ] {c34b3980-0efe-41c9-8c4f-c4bc3ae301b2} executeQuery: Code: 42. DB::Exception: Number of arguments for function greatCircleDistance doesn't match: passed 3, should be 4: In scope SELECT greatCircleDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT greatCircleDistance([], [], []);) 2025.09.25 21:10:52.674455 [ 2707 ] {c34b3980-0efe-41c9-8c4f-c4bc3ae301b2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function greatCircleDistance doesn't match: passed 3, should be 4: In scope SELECT greatCircleDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:52.756613 [ 2707 ] {1b4670c9-bc23-43e2-a64a-a19389812408} executeQuery: Code: 42. DB::Exception: Number of arguments for function greater doesn't match: passed 3, should be 2: In scope SELECT greater([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT greater([], [], []);) 2025.09.25 21:10:52.772557 [ 2707 ] {1b4670c9-bc23-43e2-a64a-a19389812408} TCPHandler: Code: 42. DB::Exception: Number of arguments for function greater doesn't match: passed 3, should be 2: In scope SELECT greater([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:52.876236 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:52.888598 [ 2707 ] {19f43280-f3e3-445e-9411-d8d28c60ce88} executeQuery: Code: 42. DB::Exception: Number of arguments for function greaterOrEquals doesn't match: passed 3, should be 2: In scope SELECT greaterOrEquals([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT greaterOrEquals([], [], []);) 2025.09.25 21:10:52.894447 [ 2707 ] {19f43280-f3e3-445e-9411-d8d28c60ce88} TCPHandler: Code: 42. DB::Exception: Number of arguments for function greaterOrEquals doesn't match: passed 3, should be 2: In scope SELECT greaterOrEquals([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:52.904219 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:53.070580 [ 2707 ] {09e09206-2572-4c97-bf64-11342f7dab7c} executeQuery: Code: 42. DB::Exception: Aggregate function groupArray requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupArray([], [], []);) 2025.09.25 21:10:53.075871 [ 2707 ] {09e09206-2572-4c97-bf64-11342f7dab7c} TCPHandler: Code: 42. DB::Exception: Aggregate function groupArray requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:53.169471 [ 2707 ] {c3d771e6-8729-49c7-b2ee-6fde350a3b78} executeQuery: Code: 42. DB::Exception: Aggregate function groupArrayInsertAt requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupArrayInsertAt([], [], []);) 2025.09.25 21:10:53.181832 [ 2707 ] {c3d771e6-8729-49c7-b2ee-6fde350a3b78} TCPHandler: Code: 42. DB::Exception: Aggregate function groupArrayInsertAt requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:53.315476 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:53.338326 [ 2707 ] {ddac5e36-2c8c-42ae-b4dd-cf76a7938227} executeQuery: Code: 42. DB::Exception: Aggregate function groupArrayIntersect requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupArrayIntersect([], [], []);) 2025.09.25 21:10:53.343404 [ 2707 ] {ddac5e36-2c8c-42ae-b4dd-cf76a7938227} TCPHandler: Code: 42. DB::Exception: Aggregate function groupArrayIntersect requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:53.446976 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:53.511834 [ 2707 ] {a706d95c-dd57-438e-9ee9-b28f666ec0dc} executeQuery: Code: 42. DB::Exception: Aggregate function groupArrayLast requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupArrayLast([], [], []);) 2025.09.25 21:10:53.519139 [ 2707 ] {a706d95c-dd57-438e-9ee9-b28f666ec0dc} TCPHandler: Code: 42. DB::Exception: Aggregate function groupArrayLast requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:53.605060 [ 2707 ] {50660931-121f-4548-898c-e5191bf65dbf} executeQuery: Code: 42. DB::Exception: Aggregate function groupArrayMovingAvg requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupArrayMovingAvg([], [], []);) 2025.09.25 21:10:53.612668 [ 2707 ] {50660931-121f-4548-898c-e5191bf65dbf} TCPHandler: Code: 42. DB::Exception: Aggregate function groupArrayMovingAvg requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:53.668011 [ 2707 ] {113ae11a-1ded-48a0-89d4-aa5ccf7b4f1e} executeQuery: Code: 42. DB::Exception: Aggregate function groupArrayMovingSum requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupArrayMovingSum([], [], []);) 2025.09.25 21:10:53.674197 [ 2707 ] {113ae11a-1ded-48a0-89d4-aa5ccf7b4f1e} TCPHandler: Code: 42. DB::Exception: Aggregate function groupArrayMovingSum requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:53.848849 [ 2707 ] {ee41b597-d020-4a3e-b894-bb482419582b} executeQuery: Code: 42. DB::Exception: Aggregate function groupArraySample requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupArraySample([], [], []);) 2025.09.25 21:10:53.855174 [ 2707 ] {ee41b597-d020-4a3e-b894-bb482419582b} TCPHandler: Code: 42. DB::Exception: Aggregate function groupArraySample requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:53.907385 [ 2707 ] {54b30bfa-c683-4376-b1e1-ee3dac7cff14} executeQuery: Code: 42. DB::Exception: Aggregate function groupArraySorted requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupArraySorted([], [], []);) 2025.09.25 21:10:53.912805 [ 2707 ] {54b30bfa-c683-4376-b1e1-ee3dac7cff14} TCPHandler: Code: 42. DB::Exception: Aggregate function groupArraySorted requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:53.970676 [ 2707 ] {3785ed17-c778-4ad3-9dd7-5104e30e1706} executeQuery: Code: 42. DB::Exception: Aggregate function groupBitAnd requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupBitAnd([], [], []);) 2025.09.25 21:10:53.975517 [ 2707 ] {3785ed17-c778-4ad3-9dd7-5104e30e1706} TCPHandler: Code: 42. DB::Exception: Aggregate function groupBitAnd requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:54.067079 [ 2707 ] {5006aaac-bd49-4b79-bf71-0fd610ed8ea2} executeQuery: Code: 42. DB::Exception: Aggregate function groupBitOr requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupBitOr([], [], []);) 2025.09.25 21:10:54.077652 [ 2707 ] {5006aaac-bd49-4b79-bf71-0fd610ed8ea2} TCPHandler: Code: 42. DB::Exception: Aggregate function groupBitOr requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:54.232022 [ 2707 ] {e58cdebd-d406-4352-a852-82138fa99c5e} executeQuery: Code: 42. DB::Exception: Aggregate function groupBitXor requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupBitXor([], [], []);) 2025.09.25 21:10:54.241110 [ 2707 ] {e58cdebd-d406-4352-a852-82138fa99c5e} TCPHandler: Code: 42. DB::Exception: Aggregate function groupBitXor requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:54.324008 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:54.327447 [ 2707 ] {87246823-dd24-4d2f-84a6-0d9604188917} executeQuery: Code: 42. DB::Exception: Aggregate function groupBitmap requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupBitmap([], [], []);) 2025.09.25 21:10:54.342903 [ 2707 ] {87246823-dd24-4d2f-84a6-0d9604188917} TCPHandler: Code: 42. DB::Exception: Aggregate function groupBitmap requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:54.364425 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:54.451218 [ 2707 ] {014cc4d2-76a8-4af1-8444-1e3bd84b8217} executeQuery: Code: 42. DB::Exception: Aggregate function groupBitmapAnd requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupBitmapAnd([], [], []);) 2025.09.25 21:10:54.459309 [ 2707 ] {014cc4d2-76a8-4af1-8444-1e3bd84b8217} TCPHandler: Code: 42. DB::Exception: Aggregate function groupBitmapAnd requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:54.560730 [ 2707 ] {46799c97-a559-4e50-9b88-d82729e88892} executeQuery: Code: 42. DB::Exception: Aggregate function groupBitmapOr requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupBitmapOr([], [], []);) 2025.09.25 21:10:54.573963 [ 2707 ] {46799c97-a559-4e50-9b88-d82729e88892} TCPHandler: Code: 42. DB::Exception: Aggregate function groupBitmapOr requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:54.668613 [ 2707 ] {49313fc8-4198-4d33-a131-f8300f0b7e0e} executeQuery: Code: 42. DB::Exception: Aggregate function groupBitmapXor requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupBitmapXor([], [], []);) 2025.09.25 21:10:54.675831 [ 2707 ] {49313fc8-4198-4d33-a131-f8300f0b7e0e} TCPHandler: Code: 42. DB::Exception: Aggregate function groupBitmapXor requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:54.711797 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:54.735864 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:54.736037 [ 2707 ] {594bdf3f-0b65-4178-ac6c-329bccdbe16e} executeQuery: Code: 42. DB::Exception: Aggregate function groupConcat requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupConcat([], [], []);) 2025.09.25 21:10:54.744816 [ 2707 ] {594bdf3f-0b65-4178-ac6c-329bccdbe16e} TCPHandler: Code: 42. DB::Exception: Aggregate function groupConcat requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:54.944526 [ 2707 ] {20b949a3-f6b7-4376-8436-a2a96a38c30d} executeQuery: Code: 42. DB::Exception: Aggregate function groupUniqArray requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT groupUniqArray([], [], []);) 2025.09.25 21:10:54.988600 [ 2707 ] {20b949a3-f6b7-4376-8436-a2a96a38c30d} TCPHandler: Code: 42. DB::Exception: Aggregate function groupUniqArray requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:55.087822 [ 2707 ] {353cf3fa-bc57-4bc6-b765-0a394b894145} executeQuery: Code: 42. DB::Exception: Aggregate function groupConcat requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT group_concat([], [], []);) 2025.09.25 21:10:55.094167 [ 2707 ] {353cf3fa-bc57-4bc6-b765-0a394b894145} TCPHandler: Code: 42. DB::Exception: Aggregate function groupConcat requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:55.201193 [ 2707 ] {ad19d625-c76c-4082-aa43-4e824e7816fb} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3CellAreaM2 doesn't match: passed 3, should be 1: In scope SELECT h3CellAreaM2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3CellAreaM2([], [], []);) 2025.09.25 21:10:55.208648 [ 2707 ] {ad19d625-c76c-4082-aa43-4e824e7816fb} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3CellAreaM2 doesn't match: passed 3, should be 1: In scope SELECT h3CellAreaM2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:55.368591 [ 2707 ] {ad45b975-ccbe-4383-acad-2b50c067f809} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3CellAreaRads2 doesn't match: passed 3, should be 1: In scope SELECT h3CellAreaRads2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3CellAreaRads2([], [], []);) 2025.09.25 21:10:55.374711 [ 2707 ] {ad45b975-ccbe-4383-acad-2b50c067f809} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3CellAreaRads2 doesn't match: passed 3, should be 1: In scope SELECT h3CellAreaRads2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:55.496800 [ 2707 ] {86626fe5-c785-4d37-8e2e-1d9813514e9f} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3Distance doesn't match: passed 3, should be 2: In scope SELECT h3Distance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3Distance([], [], []);) 2025.09.25 21:10:55.505066 [ 2707 ] {86626fe5-c785-4d37-8e2e-1d9813514e9f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3Distance doesn't match: passed 3, should be 2: In scope SELECT h3Distance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:55.575097 [ 2707 ] {7009e564-ac02-4969-bda2-2227a6b6c082} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3EdgeAngle doesn't match: passed 3, should be 1: In scope SELECT h3EdgeAngle([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3EdgeAngle([], [], []);) 2025.09.25 21:10:55.582195 [ 2707 ] {7009e564-ac02-4969-bda2-2227a6b6c082} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3EdgeAngle doesn't match: passed 3, should be 1: In scope SELECT h3EdgeAngle([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:55.677065 [ 2707 ] {3ac96e73-3517-44ba-84a3-fe7729fdccfc} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3EdgeLengthKm doesn't match: passed 3, should be 1: In scope SELECT h3EdgeLengthKm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3EdgeLengthKm([], [], []);) 2025.09.25 21:10:55.685774 [ 2707 ] {3ac96e73-3517-44ba-84a3-fe7729fdccfc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3EdgeLengthKm doesn't match: passed 3, should be 1: In scope SELECT h3EdgeLengthKm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:55.797380 [ 2707 ] {c5ddd19f-5975-4750-970e-fdf028bca3a0} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3EdgeLengthM doesn't match: passed 3, should be 1: In scope SELECT h3EdgeLengthM([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3EdgeLengthM([], [], []);) 2025.09.25 21:10:55.802983 [ 2707 ] {c5ddd19f-5975-4750-970e-fdf028bca3a0} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3EdgeLengthM doesn't match: passed 3, should be 1: In scope SELECT h3EdgeLengthM([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:55.810875 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:55.834779 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:55.872826 [ 2707 ] {76fc69d1-2083-4bac-9290-a5c2da50bb6d} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ExactEdgeLengthKm doesn't match: passed 3, should be 1: In scope SELECT h3ExactEdgeLengthKm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3ExactEdgeLengthKm([], [], []);) 2025.09.25 21:10:55.882417 [ 2707 ] {76fc69d1-2083-4bac-9290-a5c2da50bb6d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ExactEdgeLengthKm doesn't match: passed 3, should be 1: In scope SELECT h3ExactEdgeLengthKm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:55.934657 [ 2707 ] {f0bfb7f9-314b-4d53-ab30-85d73f3c28b7} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ExactEdgeLengthM doesn't match: passed 3, should be 1: In scope SELECT h3ExactEdgeLengthM([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3ExactEdgeLengthM([], [], []);) 2025.09.25 21:10:55.938760 [ 2707 ] {f0bfb7f9-314b-4d53-ab30-85d73f3c28b7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ExactEdgeLengthM doesn't match: passed 3, should be 1: In scope SELECT h3ExactEdgeLengthM([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:56.000330 [ 2707 ] {741bba4a-20db-4602-b113-1d99d7e58be5} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ExactEdgeLengthRads doesn't match: passed 3, should be 1: In scope SELECT h3ExactEdgeLengthRads([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3ExactEdgeLengthRads([], [], []);) 2025.09.25 21:10:56.005557 [ 2707 ] {741bba4a-20db-4602-b113-1d99d7e58be5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ExactEdgeLengthRads doesn't match: passed 3, should be 1: In scope SELECT h3ExactEdgeLengthRads([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:56.087114 [ 2707 ] {c5404a13-8076-4755-9210-85be70a207a1} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetBaseCell doesn't match: passed 3, should be 1: In scope SELECT h3GetBaseCell([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3GetBaseCell([], [], []);) 2025.09.25 21:10:56.093847 [ 2707 ] {c5404a13-8076-4755-9210-85be70a207a1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetBaseCell doesn't match: passed 3, should be 1: In scope SELECT h3GetBaseCell([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:56.230927 [ 2707 ] {f292312f-64ff-4a2e-a4c4-0040c6b515db} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetDestinationIndexFromUnidirectionalEdge doesn't match: passed 3, should be 1: In scope SELECT h3GetDestinationIndexFromUnidirectionalEdge([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3GetDestinationIndexFromUnidirectionalEdge([], [], []);) 2025.09.25 21:10:56.241034 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:56.247335 [ 2707 ] {f292312f-64ff-4a2e-a4c4-0040c6b515db} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetDestinationIndexFromUnidirectionalEdge doesn't match: passed 3, should be 1: In scope SELECT h3GetDestinationIndexFromUnidirectionalEdge([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:56.276304 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:56.356302 [ 2707 ] {0ed2fdde-516d-4d0f-8d47-11f49cc39b45} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetFaces doesn't match: passed 3, should be 1: In scope SELECT h3GetFaces([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3GetFaces([], [], []);) 2025.09.25 21:10:56.455061 [ 2707 ] {0ed2fdde-516d-4d0f-8d47-11f49cc39b45} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetFaces doesn't match: passed 3, should be 1: In scope SELECT h3GetFaces([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:56.520704 [ 2707 ] {24499bdd-d7c9-4022-806c-73bd6609606f} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetIndexesFromUnidirectionalEdge doesn't match: passed 3, should be 1: In scope SELECT h3GetIndexesFromUnidirectionalEdge([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3GetIndexesFromUnidirectionalEdge([], [], []);) 2025.09.25 21:10:56.524717 [ 2707 ] {24499bdd-d7c9-4022-806c-73bd6609606f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetIndexesFromUnidirectionalEdge doesn't match: passed 3, should be 1: In scope SELECT h3GetIndexesFromUnidirectionalEdge([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:56.589412 [ 2707 ] {4dbd6c7c-8b81-4cf9-8b3c-553aba20bf10} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetOriginIndexFromUnidirectionalEdge doesn't match: passed 3, should be 1: In scope SELECT h3GetOriginIndexFromUnidirectionalEdge([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3GetOriginIndexFromUnidirectionalEdge([], [], []);) 2025.09.25 21:10:56.592398 [ 2707 ] {4dbd6c7c-8b81-4cf9-8b3c-553aba20bf10} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetOriginIndexFromUnidirectionalEdge doesn't match: passed 3, should be 1: In scope SELECT h3GetOriginIndexFromUnidirectionalEdge([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:56.659492 [ 2707 ] {2ed8a4e5-5424-4b99-a343-8a5f390b2dfa} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetPentagonIndexes doesn't match: passed 3, should be 1: In scope SELECT h3GetPentagonIndexes([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3GetPentagonIndexes([], [], []);) 2025.09.25 21:10:56.662459 [ 2707 ] {2ed8a4e5-5424-4b99-a343-8a5f390b2dfa} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetPentagonIndexes doesn't match: passed 3, should be 1: In scope SELECT h3GetPentagonIndexes([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:56.744023 [ 2707 ] {df06085b-f826-4bed-b672-5015685ad9a2} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetRes0Indexes doesn't match: passed 3, should be 0: In scope SELECT h3GetRes0Indexes([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3GetRes0Indexes([], [], []);) 2025.09.25 21:10:56.750785 [ 2707 ] {df06085b-f826-4bed-b672-5015685ad9a2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetRes0Indexes doesn't match: passed 3, should be 0: In scope SELECT h3GetRes0Indexes([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:56.819867 [ 2707 ] {3d55d732-8e21-4203-99aa-f44eacf80ced} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetResolution doesn't match: passed 3, should be 1: In scope SELECT h3GetResolution([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3GetResolution([], [], []);) 2025.09.25 21:10:56.825541 [ 2707 ] {3d55d732-8e21-4203-99aa-f44eacf80ced} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetResolution doesn't match: passed 3, should be 1: In scope SELECT h3GetResolution([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:56.884125 [ 2707 ] {513e9bac-1e17-4f5c-90ba-ce1d7c241bdc} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetUnidirectionalEdge doesn't match: passed 3, should be 2: In scope SELECT h3GetUnidirectionalEdge([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3GetUnidirectionalEdge([], [], []);) 2025.09.25 21:10:56.891992 [ 2707 ] {513e9bac-1e17-4f5c-90ba-ce1d7c241bdc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetUnidirectionalEdge doesn't match: passed 3, should be 2: In scope SELECT h3GetUnidirectionalEdge([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:56.898868 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:56.919832 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:56.951745 [ 2707 ] {6bef45b6-2d2b-46cf-8db1-c11bdc6fe01f} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetUnidirectionalEdgeBoundary doesn't match: passed 3, should be 1: In scope SELECT h3GetUnidirectionalEdgeBoundary([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3GetUnidirectionalEdgeBoundary([], [], []);) 2025.09.25 21:10:56.959325 [ 2707 ] {6bef45b6-2d2b-46cf-8db1-c11bdc6fe01f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetUnidirectionalEdgeBoundary doesn't match: passed 3, should be 1: In scope SELECT h3GetUnidirectionalEdgeBoundary([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:57.011632 [ 2707 ] {a9d808d8-23ec-4341-a373-effc1c04282e} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3GetUnidirectionalEdgesFromHexagon doesn't match: passed 3, should be 1: In scope SELECT h3GetUnidirectionalEdgesFromHexagon([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3GetUnidirectionalEdgesFromHexagon([], [], []);) 2025.09.25 21:10:57.016926 [ 2707 ] {a9d808d8-23ec-4341-a373-effc1c04282e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3GetUnidirectionalEdgesFromHexagon doesn't match: passed 3, should be 1: In scope SELECT h3GetUnidirectionalEdgesFromHexagon([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:57.075284 [ 2707 ] {b9f300c6-e5df-4bce-a673-368f704dec48} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3HexAreaKm2 doesn't match: passed 3, should be 1: In scope SELECT h3HexAreaKm2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3HexAreaKm2([], [], []);) 2025.09.25 21:10:57.082195 [ 2707 ] {b9f300c6-e5df-4bce-a673-368f704dec48} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3HexAreaKm2 doesn't match: passed 3, should be 1: In scope SELECT h3HexAreaKm2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:57.140840 [ 2707 ] {fb51b533-c328-4b1d-a102-b300221cf93b} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3HexAreaM2 doesn't match: passed 3, should be 1: In scope SELECT h3HexAreaM2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3HexAreaM2([], [], []);) 2025.09.25 21:10:57.146482 [ 2707 ] {fb51b533-c328-4b1d-a102-b300221cf93b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3HexAreaM2 doesn't match: passed 3, should be 1: In scope SELECT h3HexAreaM2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:57.204628 [ 2707 ] {c884f430-3cba-4b48-b8bd-639de5ccf570} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3HexRing doesn't match: passed 3, should be 2: In scope SELECT h3HexRing([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3HexRing([], [], []);) 2025.09.25 21:10:57.210304 [ 2707 ] {c884f430-3cba-4b48-b8bd-639de5ccf570} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3HexRing doesn't match: passed 3, should be 2: In scope SELECT h3HexRing([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:57.342911 [ 2707 ] {227977fb-8d93-43c2-9d41-46793234c307} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3IndexesAreNeighbors doesn't match: passed 3, should be 2: In scope SELECT h3IndexesAreNeighbors([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3IndexesAreNeighbors([], [], []);) 2025.09.25 21:10:57.347004 [ 2707 ] {227977fb-8d93-43c2-9d41-46793234c307} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3IndexesAreNeighbors doesn't match: passed 3, should be 2: In scope SELECT h3IndexesAreNeighbors([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:57.422575 [ 7168 ] {b7a2c5dc-a653-4b59-8977-16842f3fbc49} executeQuery: Code: 41. DB::Exception: Cannot parse DateTime 2024-040: Cannot parse DateTime from String: In scope SELECT * FROM pv(timestamp_param = toDateTime('2024-040')). (CANNOT_PARSE_DATETIME) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:57080) (comment: 03146_parameterized_view_with_date.sql) (in query: select * from pv (timestamp_param=toDateTime('2024-040'));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type>, char (&) [19]) @ 0x000000001aee52b4 4. ./build_docker/./src/IO/ReadHelpers.cpp:1495: void DB::readDateTimeTextFallback(long&, DB::ReadBuffer&, DateLUTImpl const&, char const*, char const*) @ 0x000000001aee490f 5. bool DB::callOnIndexAndDataType::executeInternal(std::vector> const&, std::shared_ptr const&, unsigned long) const::'lambda'(auto const&, DB::(anonymous namespace)::BehaviourOnErrorFromString)&, DB::(anonymous namespace)::BehaviourOnErrorFromString>(DB::TypeIndex, DB::(anonymous namespace)::FunctionConvert::executeInternal(std::vector> const&, std::shared_ptr const&, unsigned long) const::'lambda'(auto const&, DB::(anonymous namespace)::BehaviourOnErrorFromString)&, DB::(anonymous namespace)::BehaviourOnErrorFromString&&) @ 0x000000000b135203 6. DB::(anonymous namespace)::FunctionConvert::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000b107d04 7. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 8. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 10. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 11. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 12. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 13. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4547: DB::QueryAnalyzer::resolveTableFunction(std::shared_ptr&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&, bool) @ 0x000000002836e44c 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x00000000283ba404 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ad09 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 20. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 21. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 24. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 25. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 27. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 28. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 29. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 2025.09.25 21:10:57.426901 [ 7168 ] {b7a2c5dc-a653-4b59-8977-16842f3fbc49} TCPHandler: Code: 41. DB::Exception: Cannot parse DateTime 2024-040: Cannot parse DateTime from String: In scope SELECT * FROM pv(timestamp_param = toDateTime('2024-040')). (CANNOT_PARSE_DATETIME), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type>, char (&) [19]) @ 0x000000001aee52b4 4. ./build_docker/./src/IO/ReadHelpers.cpp:1495: void DB::readDateTimeTextFallback(long&, DB::ReadBuffer&, DateLUTImpl const&, char const*, char const*) @ 0x000000001aee490f 5. bool DB::callOnIndexAndDataType::executeInternal(std::vector> const&, std::shared_ptr const&, unsigned long) const::'lambda'(auto const&, DB::(anonymous namespace)::BehaviourOnErrorFromString)&, DB::(anonymous namespace)::BehaviourOnErrorFromString>(DB::TypeIndex, DB::(anonymous namespace)::FunctionConvert::executeInternal(std::vector> const&, std::shared_ptr const&, unsigned long) const::'lambda'(auto const&, DB::(anonymous namespace)::BehaviourOnErrorFromString)&, DB::(anonymous namespace)::BehaviourOnErrorFromString&&) @ 0x000000000b135203 6. DB::(anonymous namespace)::FunctionConvert::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000b107d04 7. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 8. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 10. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 11. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 12. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 13. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4547: DB::QueryAnalyzer::resolveTableFunction(std::shared_ptr&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&, bool) @ 0x000000002836e44c 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x00000000283ba404 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ad09 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 20. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 21. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 24. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 25. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 27. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 28. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 29. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 2025.09.25 21:10:57.442324 [ 2707 ] {29c24994-7959-400c-b5e4-498888648dc2} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3IsPentagon doesn't match: passed 3, should be 1: In scope SELECT h3IsPentagon([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3IsPentagon([], [], []);) 2025.09.25 21:10:57.447071 [ 2707 ] {29c24994-7959-400c-b5e4-498888648dc2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3IsPentagon doesn't match: passed 3, should be 1: In scope SELECT h3IsPentagon([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:57.525381 [ 2707 ] {19d623dd-95fc-4f54-85b7-61176335a58b} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3IsResClassIII doesn't match: passed 3, should be 1: In scope SELECT h3IsResClassIII([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3IsResClassIII([], [], []);) 2025.09.25 21:10:57.536046 [ 2707 ] {19d623dd-95fc-4f54-85b7-61176335a58b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3IsResClassIII doesn't match: passed 3, should be 1: In scope SELECT h3IsResClassIII([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:57.595230 [ 2707 ] {d0dd0db0-b911-4f55-9b54-f425f08266b5} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3IsValid doesn't match: passed 3, should be 1: In scope SELECT h3IsValid([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3IsValid([], [], []);) 2025.09.25 21:10:57.607399 [ 2707 ] {d0dd0db0-b911-4f55-9b54-f425f08266b5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3IsValid doesn't match: passed 3, should be 1: In scope SELECT h3IsValid([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:57.695958 [ 2707 ] {87b2de8c-a974-4184-8cf6-f776dbad9d3e} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3Line doesn't match: passed 3, should be 2: In scope SELECT h3Line([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3Line([], [], []);) 2025.09.25 21:10:57.707907 [ 2707 ] {87b2de8c-a974-4184-8cf6-f776dbad9d3e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3Line doesn't match: passed 3, should be 2: In scope SELECT h3Line([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:57.821835 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:57.841912 [ 2707 ] {f0244040-3fd7-42ae-b672-b19fea3062fc} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3NumHexagons doesn't match: passed 3, should be 1: In scope SELECT h3NumHexagons([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3NumHexagons([], [], []);) 2025.09.25 21:10:57.852825 [ 2707 ] {f0244040-3fd7-42ae-b672-b19fea3062fc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3NumHexagons doesn't match: passed 3, should be 1: In scope SELECT h3NumHexagons([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:57.853032 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:57.892714 [ 3682 ] {3477873e-7bf9-48ec-bfd5-961b971d14ff} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:41664) (comment: 03146_parameterized_view_with_date.sql) (in query: DROP DATABASE IF EXISTS test_21vhu10q ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000291f4e6a 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 13. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 14. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 15. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 19. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 20. asan_thread_start(void*) @ 0x000000000aa43059 21. ? @ 0x00007f12cd75eac3 22. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:57.898017 [ 3682 ] {3477873e-7bf9-48ec-bfd5-961b971d14ff} DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000291f4e6a 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 13. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 14. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 15. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 19. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 20. asan_thread_start(void*) @ 0x000000000aa43059 21. ? @ 0x00007f12cd75eac3 22. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:57.954616 [ 2707 ] {4267fc20-0ee8-470b-af3e-48d58f7ab811} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3PointDistKm doesn't match: passed 3, should be 4: In scope SELECT h3PointDistKm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3PointDistKm([], [], []);) 2025.09.25 21:10:57.960037 [ 2707 ] {4267fc20-0ee8-470b-af3e-48d58f7ab811} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3PointDistKm doesn't match: passed 3, should be 4: In scope SELECT h3PointDistKm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:58.031901 [ 2707 ] {4bfe69e1-73bf-4927-881e-023dea08842f} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3PointDistM doesn't match: passed 3, should be 4: In scope SELECT h3PointDistM([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3PointDistM([], [], []);) 2025.09.25 21:10:58.046214 [ 2707 ] {4bfe69e1-73bf-4927-881e-023dea08842f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3PointDistM doesn't match: passed 3, should be 4: In scope SELECT h3PointDistM([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:58.115906 [ 2707 ] {a8eed441-8950-443a-9392-90bd89df4370} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3PointDistRads doesn't match: passed 3, should be 4: In scope SELECT h3PointDistRads([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3PointDistRads([], [], []);) 2025.09.25 21:10:58.123541 [ 2707 ] {a8eed441-8950-443a-9392-90bd89df4370} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3PointDistRads doesn't match: passed 3, should be 4: In scope SELECT h3PointDistRads([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:58.213189 [ 2707 ] {e785554e-7b93-43f6-b556-fc2fc0091e8a} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ToCenterChild doesn't match: passed 3, should be 2: In scope SELECT h3ToCenterChild([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3ToCenterChild([], [], []);) 2025.09.25 21:10:58.244414 [ 2707 ] {e785554e-7b93-43f6-b556-fc2fc0091e8a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ToCenterChild doesn't match: passed 3, should be 2: In scope SELECT h3ToCenterChild([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:58.330258 [ 2707 ] {f8ddeb61-d0c7-4e45-895d-6b83744bb46d} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ToChildren doesn't match: passed 3, should be 2: In scope SELECT h3ToChildren([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3ToChildren([], [], []);) 2025.09.25 21:10:58.338032 [ 2707 ] {f8ddeb61-d0c7-4e45-895d-6b83744bb46d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ToChildren doesn't match: passed 3, should be 2: In scope SELECT h3ToChildren([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:58.428873 [ 2707 ] {d1de92f5-c7da-42f3-96c6-bede7551109c} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ToGeo doesn't match: passed 3, should be 1: In scope SELECT h3ToGeo([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3ToGeo([], [], []);) 2025.09.25 21:10:58.435177 [ 2707 ] {d1de92f5-c7da-42f3-96c6-bede7551109c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ToGeo doesn't match: passed 3, should be 1: In scope SELECT h3ToGeo([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:58.606164 [ 2707 ] {e54c4bc5-c05d-4dbf-80ed-ac618b130e46} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ToGeoBoundary doesn't match: passed 3, should be 1: In scope SELECT h3ToGeoBoundary([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3ToGeoBoundary([], [], []);) 2025.09.25 21:10:58.617252 [ 2707 ] {e54c4bc5-c05d-4dbf-80ed-ac618b130e46} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ToGeoBoundary doesn't match: passed 3, should be 1: In scope SELECT h3ToGeoBoundary([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:58.738886 [ 2707 ] {b214c72c-483f-4fba-a74e-d59bd5c9005a} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ToParent doesn't match: passed 3, should be 2: In scope SELECT h3ToParent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3ToParent([], [], []);) 2025.09.25 21:10:58.744282 [ 2707 ] {b214c72c-483f-4fba-a74e-d59bd5c9005a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ToParent doesn't match: passed 3, should be 2: In scope SELECT h3ToParent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:58.823770 [ 2707 ] {25cf42f1-9758-4d49-83bf-3bd1c54ff953} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3ToString doesn't match: passed 3, should be 1: In scope SELECT h3ToString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3ToString([], [], []);) 2025.09.25 21:10:58.829391 [ 2707 ] {25cf42f1-9758-4d49-83bf-3bd1c54ff953} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3ToString doesn't match: passed 3, should be 1: In scope SELECT h3ToString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:58.955850 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:58.959258 [ 2707 ] {7a247ae9-0495-42d9-837c-ebee50a1f747} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3UnidirectionalEdgeIsValid doesn't match: passed 3, should be 1: In scope SELECT h3UnidirectionalEdgeIsValid([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3UnidirectionalEdgeIsValid([], [], []);) 2025.09.25 21:10:58.969654 [ 2707 ] {7a247ae9-0495-42d9-837c-ebee50a1f747} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3UnidirectionalEdgeIsValid doesn't match: passed 3, should be 1: In scope SELECT h3UnidirectionalEdgeIsValid([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:58.990878 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:59.084664 [ 2707 ] {4d2bf851-70f0-451e-a0c5-38d5a9a1adf3} executeQuery: Code: 42. DB::Exception: Number of arguments for function h3kRing doesn't match: passed 3, should be 2: In scope SELECT h3kRing([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT h3kRing([], [], []);) 2025.09.25 21:10:59.094075 [ 2707 ] {4d2bf851-70f0-451e-a0c5-38d5a9a1adf3} TCPHandler: Code: 42. DB::Exception: Number of arguments for function h3kRing doesn't match: passed 3, should be 2: In scope SELECT h3kRing([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:59.322913 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:59.333166 [ 3682 ] {9b9182b0-65d6-45d0-b26b-8378c7ac0409} executeQuery: Code: 452. DB::Exception: Setting allow_experimental_analyzer should not be changed. (SETTING_CONSTRAINT_VIOLATION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:54818) (comment: 02803_remote_cannot_clone_block.sql) (in query: SET enable_analyzer = 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges const&, DB::SettingSource) const @ 0x0000000026f807e3 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraints(DB::SettingsChanges const&, DB::SettingSource) @ 0x000000002853b6ad 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::execute() @ 0x00000000294f603b 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:59.349354 [ 3682 ] {9b9182b0-65d6-45d0-b26b-8378c7ac0409} TCPHandler: Code: 452. DB::Exception: Setting allow_experimental_analyzer should not be changed. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges const&, DB::SettingSource) const @ 0x0000000026f807e3 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraints(DB::SettingsChanges const&, DB::SettingSource) @ 0x000000002853b6ad 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::execute() @ 0x00000000294f603b 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:10:59.356564 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:10:59.472546 [ 2707 ] {acf68fe5-ae7a-4d6f-82f9-a2cbdf5d20ba} executeQuery: Code: 42. DB::Exception: Number of arguments for function has doesn't match: passed 3, should be 2: In scope SELECT has([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT has([], [], []);) 2025.09.25 21:10:59.477127 [ 2707 ] {acf68fe5-ae7a-4d6f-82f9-a2cbdf5d20ba} TCPHandler: Code: 42. DB::Exception: Number of arguments for function has doesn't match: passed 3, should be 2: In scope SELECT has([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:59.586258 [ 2707 ] {086f43db-076b-4bf8-bc86-203a7abb8368} executeQuery: Code: 42. DB::Exception: Number of arguments for function hasAll doesn't match: passed 3, should be 2: In scope SELECT hasAll([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hasAll([], [], []);) 2025.09.25 21:10:59.591482 [ 2707 ] {086f43db-076b-4bf8-bc86-203a7abb8368} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasAll doesn't match: passed 3, should be 2: In scope SELECT hasAll([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:59.684054 [ 2707 ] {0ebfc504-ce90-4c52-b0e0-f8ed9ebd38eb} executeQuery: Code: 42. DB::Exception: Number of arguments for function hasAny doesn't match: passed 3, should be 2: In scope SELECT hasAny([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hasAny([], [], []);) 2025.09.25 21:10:59.690142 [ 2707 ] {0ebfc504-ce90-4c52-b0e0-f8ed9ebd38eb} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasAny doesn't match: passed 3, should be 2: In scope SELECT hasAny([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:10:59.826598 [ 2707 ] {42234aa1-0028-44c1-b797-03110fa004cd} executeQuery: Code: 43. DB::Exception: First argument for function hasColumnInTable must be const String.: In scope SELECT hasColumnInTable([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hasColumnInTable([], [], []);) 2025.09.25 21:10:59.834759 [ 2707 ] {42234aa1-0028-44c1-b797-03110fa004cd} TCPHandler: Code: 43. DB::Exception: First argument for function hasColumnInTable must be const String.: In scope SELECT hasColumnInTable([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:10:59.920332 [ 2707 ] {6dfef80d-f250-44d0-9818-370ed0274ef2} executeQuery: Code: 42. DB::Exception: Number of arguments for function hasSubsequence doesn't match: passed 3, should be 2: In scope SELECT hasSubsequence([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hasSubsequence([], [], []);) 2025.09.25 21:10:59.935771 [ 2707 ] {6dfef80d-f250-44d0-9818-370ed0274ef2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasSubsequence doesn't match: passed 3, should be 2: In scope SELECT hasSubsequence([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:00.036821 [ 2707 ] {84303291-a86f-4720-813d-9c0633e5a07a} executeQuery: Code: 42. DB::Exception: Number of arguments for function hasSubsequenceCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT hasSubsequenceCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hasSubsequenceCaseInsensitive([], [], []);) 2025.09.25 21:11:00.041591 [ 2707 ] {84303291-a86f-4720-813d-9c0633e5a07a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasSubsequenceCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT hasSubsequenceCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:00.094634 [ 2707 ] {a671388d-e416-4ffa-be67-5cd349759287} executeQuery: Code: 42. DB::Exception: Number of arguments for function hasSubsequenceCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT hasSubsequenceCaseInsensitiveUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hasSubsequenceCaseInsensitiveUTF8([], [], []);) 2025.09.25 21:11:00.103927 [ 2707 ] {a671388d-e416-4ffa-be67-5cd349759287} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasSubsequenceCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT hasSubsequenceCaseInsensitiveUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:00.204768 [ 2707 ] {2fbf9020-1e97-49ac-b13c-2b27404f8e5f} executeQuery: Code: 42. DB::Exception: Number of arguments for function hasSubsequenceUTF8 doesn't match: passed 3, should be 2: In scope SELECT hasSubsequenceUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hasSubsequenceUTF8([], [], []);) 2025.09.25 21:11:00.210492 [ 2707 ] {2fbf9020-1e97-49ac-b13c-2b27404f8e5f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasSubsequenceUTF8 doesn't match: passed 3, should be 2: In scope SELECT hasSubsequenceUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:00.333418 [ 2707 ] {a807514a-3e17-4353-905c-b8a1b412c4da} executeQuery: Code: 42. DB::Exception: Number of arguments for function hasSubstr doesn't match: passed 3, should be 2: In scope SELECT hasSubstr([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hasSubstr([], [], []);) 2025.09.25 21:11:00.337995 [ 2707 ] {a807514a-3e17-4353-905c-b8a1b412c4da} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasSubstr doesn't match: passed 3, should be 2: In scope SELECT hasSubstr([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:00.357075 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:00.372180 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:00.390808 [ 2707 ] {21739e27-37f5-4c8f-8869-5dfcffa75f96} executeQuery: Code: 42. DB::Exception: Number of arguments for function hasThreadFuzzer doesn't match: passed 3, should be 0: In scope SELECT hasThreadFuzzer([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hasThreadFuzzer([], [], []);) 2025.09.25 21:11:00.399342 [ 2707 ] {21739e27-37f5-4c8f-8869-5dfcffa75f96} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasThreadFuzzer doesn't match: passed 3, should be 0: In scope SELECT hasThreadFuzzer([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:00.440527 [ 2707 ] {448b4d35-d710-4a0e-b9be-c35fae076820} executeQuery: Code: 42. DB::Exception: Number of arguments for function hasToken doesn't match: passed 3, should be 2: In scope SELECT hasToken([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hasToken([], [], []);) 2025.09.25 21:11:00.452747 [ 2707 ] {448b4d35-d710-4a0e-b9be-c35fae076820} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasToken doesn't match: passed 3, should be 2: In scope SELECT hasToken([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:00.548062 [ 2707 ] {c07ee793-2257-4915-bf3b-d2b184571e6c} executeQuery: Code: 42. DB::Exception: Number of arguments for function hasTokenCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT hasTokenCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hasTokenCaseInsensitive([], [], []);) 2025.09.25 21:11:00.553869 [ 2707 ] {c07ee793-2257-4915-bf3b-d2b184571e6c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasTokenCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT hasTokenCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:00.608442 [ 2707 ] {9606fafc-5845-4343-ae57-c95970759aca} executeQuery: Code: 42. DB::Exception: Number of arguments for function hasTokenCaseInsensitiveOrNull doesn't match: passed 3, should be 2: In scope SELECT hasTokenCaseInsensitiveOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hasTokenCaseInsensitiveOrNull([], [], []);) 2025.09.25 21:11:00.623608 [ 2707 ] {9606fafc-5845-4343-ae57-c95970759aca} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasTokenCaseInsensitiveOrNull doesn't match: passed 3, should be 2: In scope SELECT hasTokenCaseInsensitiveOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:00.706691 [ 2707 ] {18e5e334-4033-4274-abbd-139d35da060a} executeQuery: Code: 42. DB::Exception: Number of arguments for function hasTokenOrNull doesn't match: passed 3, should be 2: In scope SELECT hasTokenOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hasTokenOrNull([], [], []);) 2025.09.25 21:11:00.715728 [ 2707 ] {18e5e334-4033-4274-abbd-139d35da060a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hasTokenOrNull doesn't match: passed 3, should be 2: In scope SELECT hasTokenOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:00.837442 [ 2707 ] {c0be828e-4c18-46a9-9bff-fc38d82477b0} executeQuery: Code: 42. DB::Exception: Number of arguments for function hex doesn't match: passed 3, should be 1: In scope SELECT hex([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hex([], [], []);) 2025.09.25 21:11:00.840879 [ 2707 ] {c0be828e-4c18-46a9-9bff-fc38d82477b0} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hex doesn't match: passed 3, should be 1: In scope SELECT hex([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:00.928795 [ 2707 ] {b3f8a0b9-c1d6-4114-99b2-049f93715503} executeQuery: Code: 42. DB::Exception: Number of arguments for function hilbertDecode doesn't match: passed 3, should be 2: In scope SELECT hilbertDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hilbertDecode([], [], []);) 2025.09.25 21:11:00.933253 [ 2707 ] {b3f8a0b9-c1d6-4114-99b2-049f93715503} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hilbertDecode doesn't match: passed 3, should be 2: In scope SELECT hilbertDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:01.016209 [ 2707 ] {1b691a81-6efd-456c-b9bf-338403389c48} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for function hilbertEncode, should be a native UInt: In scope SELECT hilbertEncode([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hilbertEncode([], [], []);) 2025.09.25 21:11:01.024677 [ 2707 ] {1b691a81-6efd-456c-b9bf-338403389c48} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for function hilbertEncode, should be a native UInt: In scope SELECT hilbertEncode([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:01.143434 [ 2707 ] {4f755fd2-dc79-41e5-8958-397bb513ba46} executeQuery: Code: 42. DB::Exception: Function histogram requires single parameter: bins count. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT histogram([], [], []);) 2025.09.25 21:11:01.146676 [ 2707 ] {4f755fd2-dc79-41e5-8958-397bb513ba46} TCPHandler: Code: 42. DB::Exception: Function histogram requires single parameter: bins count. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:01.219593 [ 2707 ] {c2e7eb8d-d768-4fc7-98c9-714bf26899a2} executeQuery: Code: 48. DB::Exception: Hive hash is not combineable for multiple arguments: In scope SELECT hiveHash([], [], []). (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hiveHash([], [], []);) 2025.09.25 21:11:01.226776 [ 2707 ] {c2e7eb8d-d768-4fc7-98c9-714bf26899a2} TCPHandler: Code: 48. DB::Exception: Hive hash is not combineable for multiple arguments: In scope SELECT hiveHash([], [], []). (NOT_IMPLEMENTED) 2025.09.25 21:11:01.274069 [ 7168 ] {8a407b97-4e80-4b89-97e1-78d2e6d992f2} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:54826) (comment: 00963_startsWith_force_primary_key.sql) (in query: SELECT count() from test_startsWith where startsWith(a, 'abc') settings force_primary_key=1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:11:01.277872 [ 7168 ] {8a407b97-4e80-4b89-97e1-78d2e6d992f2} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:11:01.338352 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:01.382170 [ 2707 ] {51078597-2d86-45f4-a825-e84ee8e0b95c} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function hop. Should be a date with time: In scope SELECT hop([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hop([], [], []);) 2025.09.25 21:11:01.400528 [ 2707 ] {51078597-2d86-45f4-a825-e84ee8e0b95c} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function hop. Should be a date with time: In scope SELECT hop([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:01.423103 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:01.492977 [ 2707 ] {325d7ca7-4d2c-45e4-baad-665af98ea158} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function hopEnd. Should be a date with time: In scope SELECT hopEnd([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hopEnd([], [], []);) 2025.09.25 21:11:01.497590 [ 2707 ] {325d7ca7-4d2c-45e4-baad-665af98ea158} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function hopEnd. Should be a date with time: In scope SELECT hopEnd([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:01.563121 [ 2707 ] {d984712f-35ed-4543-8e87-44b5e9bf9d0d} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function hopStart. Should be a date with time: In scope SELECT hopStart([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hopStart([], [], []);) 2025.09.25 21:11:01.567827 [ 2707 ] {d984712f-35ed-4543-8e87-44b5e9bf9d0d} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function hopStart. Should be a date with time: In scope SELECT hopStart([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:01.623933 [ 2707 ] {104f1c41-d3ef-4fa0-9ee4-1712d3c5da37} executeQuery: Code: 42. DB::Exception: Number of arguments for function hostName doesn't match: passed 3, should be 0: In scope SELECT hostName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hostName([], [], []);) 2025.09.25 21:11:01.628980 [ 2707 ] {104f1c41-d3ef-4fa0-9ee4-1712d3c5da37} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hostName doesn't match: passed 3, should be 0: In scope SELECT hostName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:01.676889 [ 2707 ] {9d06085e-9ce8-4f23-b068-939e57b7b791} executeQuery: Code: 42. DB::Exception: Number of arguments for function hostName doesn't match: passed 3, should be 0: In scope SELECT hostname([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hostname([], [], []);) 2025.09.25 21:11:01.691195 [ 3682 ] {ee771ed7-330f-4d70-9bfc-5de458b77124} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:41696) (comment: 00963_startsWith_force_primary_key.sql) (in query: DROP DATABASE IF EXISTS test_u8ptcjcy ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:11:01.698134 [ 3682 ] {ee771ed7-330f-4d70-9bfc-5de458b77124} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:01.754781 [ 2707 ] {9d06085e-9ce8-4f23-b068-939e57b7b791} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hostName doesn't match: passed 3, should be 0: In scope SELECT hostname([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:01.791076 [ 2707 ] {0afba19a-a5b6-47cf-a4c3-6b2b5c5cf276} executeQuery: Code: 42. DB::Exception: Number of arguments for function hypot doesn't match: passed 3, should be 2: In scope SELECT hypot([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT hypot([], [], []);) 2025.09.25 21:11:01.793880 [ 2707 ] {0afba19a-a5b6-47cf-a4c3-6b2b5c5cf276} TCPHandler: Code: 42. DB::Exception: Number of arguments for function hypot doesn't match: passed 3, should be 2: In scope SELECT hypot([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:01.852860 [ 2707 ] {f7f727cb-3709-4d04-ab6e-0ac79b0e639e} executeQuery: Code: 42. DB::Exception: Number of arguments for function identity doesn't match: passed 3, should be 1: In scope SELECT identity([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT identity([], [], []);) 2025.09.25 21:11:01.858318 [ 2707 ] {f7f727cb-3709-4d04-ab6e-0ac79b0e639e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function identity doesn't match: passed 3, should be 1: In scope SELECT identity([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:01.982661 [ 2707 ] {bc1211ae-af04-4f9e-ac57-586e058d3665} executeQuery: Code: 42. DB::Exception: Number of arguments for function idnaDecode doesn't match: passed 3, should be 1: In scope SELECT idnaDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT idnaDecode([], [], []);) 2025.09.25 21:11:01.987810 [ 2707 ] {bc1211ae-af04-4f9e-ac57-586e058d3665} TCPHandler: Code: 42. DB::Exception: Number of arguments for function idnaDecode doesn't match: passed 3, should be 1: In scope SELECT idnaDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:02.072385 [ 2707 ] {f5355a9e-c387-470a-a7cf-4400fe791bc0} executeQuery: Code: 42. DB::Exception: Number of arguments for function idnaEncode doesn't match: passed 3, should be 1: In scope SELECT idnaEncode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT idnaEncode([], [], []);) 2025.09.25 21:11:02.089515 [ 2707 ] {f5355a9e-c387-470a-a7cf-4400fe791bc0} TCPHandler: Code: 42. DB::Exception: Number of arguments for function idnaEncode doesn't match: passed 3, should be 1: In scope SELECT idnaEncode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:02.228532 [ 2707 ] {667834a2-876a-4cd0-bf72-1106be2ef1c9} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument (condition) of function if. Must be UInt8.: In scope SELECT if([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT if([], [], []);) 2025.09.25 21:11:02.236137 [ 2707 ] {667834a2-876a-4cd0-bf72-1106be2ef1c9} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument (condition) of function if. Must be UInt8.: In scope SELECT if([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:02.400013 [ 2707 ] {4ff7d6d2-54cf-4eb4-bc43-01d4b8676e32} executeQuery: Code: 42. DB::Exception: Number of arguments for function ifNotFinite doesn't match: passed 3, should be 2: In scope SELECT ifNotFinite([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ifNotFinite([], [], []);) 2025.09.25 21:11:02.409616 [ 2707 ] {4ff7d6d2-54cf-4eb4-bc43-01d4b8676e32} TCPHandler: Code: 42. DB::Exception: Number of arguments for function ifNotFinite doesn't match: passed 3, should be 2: In scope SELECT ifNotFinite([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:02.480012 [ 2707 ] {9e9d62fe-44f4-4ec7-afbe-4a4cc5860314} executeQuery: Code: 42. DB::Exception: Number of arguments for function ifNull doesn't match: passed 3, should be 2: In scope SELECT ifNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ifNull([], [], []);) 2025.09.25 21:11:02.492703 [ 2707 ] {9e9d62fe-44f4-4ec7-afbe-4a4cc5860314} TCPHandler: Code: 42. DB::Exception: Number of arguments for function ifNull doesn't match: passed 3, should be 2: In scope SELECT ifNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:02.597818 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:02.627140 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:02.717532 [ 2707 ] {7a6b0922-16c4-424d-a6f5-7fc7a719c656} executeQuery: Code: 42. DB::Exception: Number of arguments for function ilike doesn't match: passed 3, should be 2: In scope SELECT ilike([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ilike([], [], []);) 2025.09.25 21:11:02.739047 [ 2707 ] {7a6b0922-16c4-424d-a6f5-7fc7a719c656} TCPHandler: Code: 42. DB::Exception: Number of arguments for function ilike doesn't match: passed 3, should be 2: In scope SELECT ilike([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:02.840310 [ 2707 ] {7ae293c2-83a4-42bf-9ebe-594d865cceeb} executeQuery: Code: 42. DB::Exception: Function 'in' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT in([], [], []);) 2025.09.25 21:11:02.848967 [ 2707 ] {7ae293c2-83a4-42bf-9ebe-594d865cceeb} TCPHandler: Code: 42. DB::Exception: Function 'in' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:03.077147 [ 2707 ] {eb0f4b85-7006-4084-ab10-d36ccf92e5aa} executeQuery: Code: 42. DB::Exception: Function 'inIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT inIgnoreSet([], [], []);) 2025.09.25 21:11:03.081241 [ 2707 ] {eb0f4b85-7006-4084-ab10-d36ccf92e5aa} TCPHandler: Code: 42. DB::Exception: Function 'inIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:03.387168 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:03.424936 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:03.445613 [ 2707 ] {a439c22d-fb88-4925-95a9-50dd23c7f0ae} executeQuery: Code: 42. DB::Exception: Number of arguments for function indexOf doesn't match: passed 3, should be 2: In scope SELECT indexOf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT indexOf([], [], []);) 2025.09.25 21:11:03.455725 [ 2707 ] {a439c22d-fb88-4925-95a9-50dd23c7f0ae} TCPHandler: Code: 42. DB::Exception: Number of arguments for function indexOf doesn't match: passed 3, should be 2: In scope SELECT indexOf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:03.565809 [ 2707 ] {d16eb974-f28d-4661-8ff2-3005b5cb79ff} executeQuery: Code: 42. DB::Exception: Number of arguments for function initcap doesn't match: passed 3, should be 1: In scope SELECT initcap([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT initcap([], [], []);) 2025.09.25 21:11:03.581262 [ 2707 ] {d16eb974-f28d-4661-8ff2-3005b5cb79ff} TCPHandler: Code: 42. DB::Exception: Number of arguments for function initcap doesn't match: passed 3, should be 1: In scope SELECT initcap([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:03.683434 [ 2707 ] {f085f16c-4410-47ae-9e96-8e6cce1872ad} executeQuery: Code: 42. DB::Exception: Number of arguments for function initcapUTF8 doesn't match: passed 3, should be 1: In scope SELECT initcapUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT initcapUTF8([], [], []);) 2025.09.25 21:11:03.689266 [ 2707 ] {f085f16c-4410-47ae-9e96-8e6cce1872ad} TCPHandler: Code: 42. DB::Exception: Number of arguments for function initcapUTF8 doesn't match: passed 3, should be 1: In scope SELECT initcapUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:03.833652 [ 2707 ] {f1f3a903-12b5-4f28-a3d9-d8ab3b715952} executeQuery: Code: 42. DB::Exception: Number of arguments for function initialQueryID doesn't match: passed 3, should be 0: In scope SELECT initialQueryID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT initialQueryID([], [], []);) 2025.09.25 21:11:03.850248 [ 2707 ] {f1f3a903-12b5-4f28-a3d9-d8ab3b715952} TCPHandler: Code: 42. DB::Exception: Number of arguments for function initialQueryID doesn't match: passed 3, should be 0: In scope SELECT initialQueryID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:03.850456 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:03.915547 [ 2707 ] {3d0b59a7-0ad1-4a29-afc7-1d903a529473} executeQuery: Code: 42. DB::Exception: Number of arguments for function initialQueryID doesn't match: passed 3, should be 0: In scope SELECT initial_query_id([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT initial_query_id([], [], []);) 2025.09.25 21:11:03.923083 [ 2707 ] {3d0b59a7-0ad1-4a29-afc7-1d903a529473} TCPHandler: Code: 42. DB::Exception: Number of arguments for function initialQueryID doesn't match: passed 3, should be 0: In scope SELECT initial_query_id([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:03.938437 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:04.008914 [ 2707 ] {784c5458-fa1a-435d-8e9f-9830ae98ead3} executeQuery: Code: 43. DB::Exception: First argument for function initializeAggregation must be constant string: name of aggregate function.: In scope SELECT initializeAggregation([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT initializeAggregation([], [], []);) 2025.09.25 21:11:04.017332 [ 2707 ] {784c5458-fa1a-435d-8e9f-9830ae98ead3} TCPHandler: Code: 43. DB::Exception: First argument for function initializeAggregation must be constant string: name of aggregate function.: In scope SELECT initializeAggregation([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:04.068963 [ 2707 ] {6ceb82d9-9f8c-418c-8abb-1f08bb537220} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function positionCaseInsensitive: In scope SELECT instr([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT instr([], [], []);) 2025.09.25 21:11:04.073756 [ 2707 ] {6ceb82d9-9f8c-418c-8abb-1f08bb537220} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function positionCaseInsensitive: In scope SELECT instr([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:04.123122 [ 3682 ] {} TCPHandler: Code: 452. DB::Exception: Setting allow_experimental_analyzer should not be changed. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/remove_if.h:32: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f80932 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./build_docker/./src/Server/TCPHandler.cpp:2034: DB::TCPHandler::receiveQuery() @ 0x000000002d1a17ad 10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::receivePacket() @ 0x000000002d1877e2 11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d175061 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:11:04.168093 [ 2707 ] {c3a9e81d-712e-4494-9d98-6f0974a10c46} executeQuery: Code: 42. DB::Exception: Number of arguments for function intDiv doesn't match: passed 3, should be 2: In scope SELECT intDiv([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT intDiv([], [], []);) 2025.09.25 21:11:04.176385 [ 2707 ] {c3a9e81d-712e-4494-9d98-6f0974a10c46} TCPHandler: Code: 42. DB::Exception: Number of arguments for function intDiv doesn't match: passed 3, should be 2: In scope SELECT intDiv([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:04.321198 [ 2707 ] {a03b9f41-706c-4eb8-ab66-304279097c5a} executeQuery: Code: 42. DB::Exception: Number of arguments for function intDivOrZero doesn't match: passed 3, should be 2: In scope SELECT intDivOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT intDivOrZero([], [], []);) 2025.09.25 21:11:04.326637 [ 2707 ] {a03b9f41-706c-4eb8-ab66-304279097c5a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function intDivOrZero doesn't match: passed 3, should be 2: In scope SELECT intDivOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:04.454999 [ 2707 ] {097526aa-6fd9-47aa-a7e8-6ba4f2c057fe} executeQuery: Code: 42. DB::Exception: Number of arguments for function intExp10 doesn't match: passed 3, should be 1: In scope SELECT intExp10([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT intExp10([], [], []);) 2025.09.25 21:11:04.463751 [ 2707 ] {097526aa-6fd9-47aa-a7e8-6ba4f2c057fe} TCPHandler: Code: 42. DB::Exception: Number of arguments for function intExp10 doesn't match: passed 3, should be 1: In scope SELECT intExp10([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:04.586070 [ 2707 ] {8ec57beb-70e0-497f-8005-e0e91a86e624} executeQuery: Code: 42. DB::Exception: Number of arguments for function intExp2 doesn't match: passed 3, should be 1: In scope SELECT intExp2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT intExp2([], [], []);) 2025.09.25 21:11:04.596340 [ 2707 ] {8ec57beb-70e0-497f-8005-e0e91a86e624} TCPHandler: Code: 42. DB::Exception: Number of arguments for function intExp2 doesn't match: passed 3, should be 1: In scope SELECT intExp2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:04.684589 [ 2707 ] {7545379b-aaf7-45aa-a80c-5405cf9145a8} executeQuery: Code: 42. DB::Exception: Number of arguments for function intHash32 doesn't match: passed 3, should be 1: In scope SELECT intHash32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT intHash32([], [], []);) 2025.09.25 21:11:04.688492 [ 2707 ] {7545379b-aaf7-45aa-a80c-5405cf9145a8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function intHash32 doesn't match: passed 3, should be 1: In scope SELECT intHash32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:04.802165 [ 2707 ] {c069bc48-1413-4ca0-910a-ed18b3d36692} executeQuery: Code: 42. DB::Exception: Number of arguments for function intHash64 doesn't match: passed 3, should be 1: In scope SELECT intHash64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT intHash64([], [], []);) 2025.09.25 21:11:04.810480 [ 2707 ] {c069bc48-1413-4ca0-910a-ed18b3d36692} TCPHandler: Code: 42. DB::Exception: Number of arguments for function intHash64 doesn't match: passed 3, should be 1: In scope SELECT intHash64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:04.892990 [ 2707 ] {6ca72e55-2e25-4e10-9c0d-34fdd90bbb66} executeQuery: Code: 42. DB::Exception: Aggregate function intervalLengthSum requires two timestamps argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT intervalLengthSum([], [], []);) 2025.09.25 21:11:04.904044 [ 2707 ] {6ca72e55-2e25-4e10-9c0d-34fdd90bbb66} TCPHandler: Code: 42. DB::Exception: Aggregate function intervalLengthSum requires two timestamps argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:04.972350 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:05.009761 [ 2707 ] {deec87fa-9802-4961-90f7-0870c2df9cec} executeQuery: Code: 42. DB::Exception: Number of arguments for function isConstant doesn't match: passed 3, should be 1: In scope SELECT isConstant([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT isConstant([], [], []);) 2025.09.25 21:11:05.026382 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:05.026237 [ 2707 ] {deec87fa-9802-4961-90f7-0870c2df9cec} TCPHandler: Code: 42. DB::Exception: Number of arguments for function isConstant doesn't match: passed 3, should be 1: In scope SELECT isConstant([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:05.104267 [ 2707 ] {9504e186-aebc-4128-bd2d-c081842263f4} executeQuery: Code: 42. DB::Exception: Number of arguments for function isDecimalOverflow doesn't match: passed 3, should be 1 or 2.: In scope SELECT isDecimalOverflow([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT isDecimalOverflow([], [], []);) 2025.09.25 21:11:05.109926 [ 2707 ] {9504e186-aebc-4128-bd2d-c081842263f4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function isDecimalOverflow doesn't match: passed 3, should be 1 or 2.: In scope SELECT isDecimalOverflow([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:05.194633 [ 2707 ] {34d8beb8-cb74-46e6-ae0d-00b377a0d6a8} executeQuery: Code: 42. DB::Exception: Number of arguments for function isDynamicElementInSharedData doesn't match: passed 3, should be 1: In scope SELECT isDynamicElementInSharedData([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT isDynamicElementInSharedData([], [], []);) 2025.09.25 21:11:05.201190 [ 2707 ] {34d8beb8-cb74-46e6-ae0d-00b377a0d6a8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function isDynamicElementInSharedData doesn't match: passed 3, should be 1: In scope SELECT isDynamicElementInSharedData([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:05.293637 [ 2707 ] {3c5e979e-fbf9-4319-93a4-20e11a0906f7} executeQuery: Code: 42. DB::Exception: Number of arguments for function isFinite doesn't match: passed 3, should be 1: In scope SELECT isFinite([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT isFinite([], [], []);) 2025.09.25 21:11:05.313470 [ 2707 ] {3c5e979e-fbf9-4319-93a4-20e11a0906f7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function isFinite doesn't match: passed 3, should be 1: In scope SELECT isFinite([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:05.414312 [ 2707 ] {780d1666-b053-4043-af9b-a7db55f4404c} executeQuery: Code: 42. DB::Exception: Number of arguments for function isIPAddressInRange doesn't match: passed 3, should be 2: In scope SELECT isIPAddressInRange([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT isIPAddressInRange([], [], []);) 2025.09.25 21:11:05.426953 [ 2707 ] {780d1666-b053-4043-af9b-a7db55f4404c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function isIPAddressInRange doesn't match: passed 3, should be 2: In scope SELECT isIPAddressInRange([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:05.534229 [ 2707 ] {4e37399e-562f-4931-91f9-708cb37df3ef} executeQuery: Code: 42. DB::Exception: Number of arguments for function isIPv4String doesn't match: passed 3, should be 1: In scope SELECT isIPv4String([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT isIPv4String([], [], []);) 2025.09.25 21:11:05.545268 [ 2707 ] {4e37399e-562f-4931-91f9-708cb37df3ef} TCPHandler: Code: 42. DB::Exception: Number of arguments for function isIPv4String doesn't match: passed 3, should be 1: In scope SELECT isIPv4String([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:05.630401 [ 2707 ] {c59486a9-2d9c-4632-bfdf-ee4d7fe45990} executeQuery: Code: 42. DB::Exception: Number of arguments for function isIPv6String doesn't match: passed 3, should be 1: In scope SELECT isIPv6String([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT isIPv6String([], [], []);) 2025.09.25 21:11:05.636586 [ 2707 ] {c59486a9-2d9c-4632-bfdf-ee4d7fe45990} TCPHandler: Code: 42. DB::Exception: Number of arguments for function isIPv6String doesn't match: passed 3, should be 1: In scope SELECT isIPv6String([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:05.715651 [ 2707 ] {07a77e04-02ab-457d-91fc-4b6e621ca410} executeQuery: Code: 42. DB::Exception: Number of arguments for function isInfinite doesn't match: passed 3, should be 1: In scope SELECT isInfinite([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT isInfinite([], [], []);) 2025.09.25 21:11:05.722449 [ 2707 ] {07a77e04-02ab-457d-91fc-4b6e621ca410} TCPHandler: Code: 42. DB::Exception: Number of arguments for function isInfinite doesn't match: passed 3, should be 1: In scope SELECT isInfinite([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:05.807898 [ 2707 ] {0ef753e9-8e22-4beb-bfd7-7517fdc27823} executeQuery: Code: 42. DB::Exception: Number of arguments for function isNaN doesn't match: passed 3, should be 1: In scope SELECT isNaN([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT isNaN([], [], []);) 2025.09.25 21:11:05.815127 [ 2707 ] {0ef753e9-8e22-4beb-bfd7-7517fdc27823} TCPHandler: Code: 42. DB::Exception: Number of arguments for function isNaN doesn't match: passed 3, should be 1: In scope SELECT isNaN([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:05.856597 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:05.889032 [ 2707 ] {f3e14a05-5e93-4774-ac0e-9c607844f5f3} executeQuery: Code: 42. DB::Exception: Number of arguments for function isNotDistinctFrom doesn't match: passed 3, should be 2: In scope SELECT isNotDistinctFrom([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT isNotDistinctFrom([], [], []);) 2025.09.25 21:11:05.898074 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:05.905087 [ 2707 ] {f3e14a05-5e93-4774-ac0e-9c607844f5f3} TCPHandler: Code: 42. DB::Exception: Number of arguments for function isNotDistinctFrom doesn't match: passed 3, should be 2: In scope SELECT isNotDistinctFrom([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:05.961916 [ 2707 ] {95b297a6-da62-460d-af65-345f2638a6e7} executeQuery: Code: 42. DB::Exception: Number of arguments for function isNotNull doesn't match: passed 3, should be 1: In scope SELECT isNotNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT isNotNull([], [], []);) 2025.09.25 21:11:05.982504 [ 2707 ] {95b297a6-da62-460d-af65-345f2638a6e7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function isNotNull doesn't match: passed 3, should be 1: In scope SELECT isNotNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:06.088152 [ 2707 ] {3fcc85fa-6a78-488f-9f25-f14521b71bdd} executeQuery: Code: 42. DB::Exception: Number of arguments for function isNull doesn't match: passed 3, should be 1: In scope SELECT isNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT isNull([], [], []);) 2025.09.25 21:11:06.109707 [ 2707 ] {3fcc85fa-6a78-488f-9f25-f14521b71bdd} TCPHandler: Code: 42. DB::Exception: Number of arguments for function isNull doesn't match: passed 3, should be 1: In scope SELECT isNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:06.189397 [ 2707 ] {85e3eefe-a711-41d6-9846-e148ae69fc7c} executeQuery: Code: 42. DB::Exception: Number of arguments for function isNullable doesn't match: passed 3, should be 1: In scope SELECT isNullable([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT isNullable([], [], []);) 2025.09.25 21:11:06.198007 [ 2707 ] {85e3eefe-a711-41d6-9846-e148ae69fc7c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function isNullable doesn't match: passed 3, should be 1: In scope SELECT isNullable([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:06.289728 [ 2707 ] {dd6865bf-f0e3-4485-a411-44ee4fe4b101} executeQuery: Code: 42. DB::Exception: Function isValidJSON needs exactly one argument: In scope SELECT isValidJSON([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT isValidJSON([], [], []);) 2025.09.25 21:11:06.296032 [ 2707 ] {dd6865bf-f0e3-4485-a411-44ee4fe4b101} TCPHandler: Code: 42. DB::Exception: Function isValidJSON needs exactly one argument: In scope SELECT isValidJSON([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:06.345000 [ 2707 ] {71a60a5e-e7a9-4745-802d-28037aa49f62} executeQuery: Code: 42. DB::Exception: Number of arguments for function isValidUTF8 doesn't match: passed 3, should be 1: In scope SELECT isValidUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT isValidUTF8([], [], []);) 2025.09.25 21:11:06.348841 [ 2707 ] {71a60a5e-e7a9-4745-802d-28037aa49f62} TCPHandler: Code: 42. DB::Exception: Number of arguments for function isValidUTF8 doesn't match: passed 3, should be 1: In scope SELECT isValidUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:06.456668 [ 2707 ] {65345574-80f3-48e3-8ce7-47a3d4ed0f62} executeQuery: Code: 42. DB::Exception: Number of arguments for function isZeroOrNull doesn't match: passed 3, should be 1: In scope SELECT isZeroOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT isZeroOrNull([], [], []);) 2025.09.25 21:11:06.464824 [ 2707 ] {65345574-80f3-48e3-8ce7-47a3d4ed0f62} TCPHandler: Code: 42. DB::Exception: Number of arguments for function isZeroOrNull doesn't match: passed 3, should be 1: In scope SELECT isZeroOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:06.560672 [ 2707 ] {53f86f45-f242-4871-a6bb-87d2c6ccc36a} executeQuery: Code: 42. DB::Exception: Number of arguments for function jaroSimilarity doesn't match: passed 3, should be 2: In scope SELECT jaroSimilarity([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT jaroSimilarity([], [], []);) 2025.09.25 21:11:06.565498 [ 2707 ] {53f86f45-f242-4871-a6bb-87d2c6ccc36a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function jaroSimilarity doesn't match: passed 3, should be 2: In scope SELECT jaroSimilarity([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:06.644677 [ 2707 ] {6feff0bd-77b3-4f7c-86c0-01922ab98647} executeQuery: Code: 42. DB::Exception: Number of arguments for function jaroWinklerSimilarity doesn't match: passed 3, should be 2: In scope SELECT jaroWinklerSimilarity([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT jaroWinklerSimilarity([], [], []);) 2025.09.25 21:11:06.649167 [ 2707 ] {6feff0bd-77b3-4f7c-86c0-01922ab98647} TCPHandler: Code: 42. DB::Exception: Number of arguments for function jaroWinklerSimilarity doesn't match: passed 3, should be 2: In scope SELECT jaroWinklerSimilarity([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:06.720904 [ 2707 ] {97d78ab5-65f5-44ad-aecc-20c33adafc1d} executeQuery: Code: 48. DB::Exception: Java hash is not combineable for multiple arguments: In scope SELECT javaHash([], [], []). (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT javaHash([], [], []);) 2025.09.25 21:11:06.723732 [ 2707 ] {97d78ab5-65f5-44ad-aecc-20c33adafc1d} TCPHandler: Code: 48. DB::Exception: Java hash is not combineable for multiple arguments: In scope SELECT javaHash([], [], []). (NOT_IMPLEMENTED) 2025.09.25 21:11:06.796047 [ 2707 ] {99710493-db6e-4c52-b287-c9a4577ee071} executeQuery: Code: 48. DB::Exception: Java hash is not combineable for multiple arguments: In scope SELECT javaHashUTF16LE([], [], []). (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT javaHashUTF16LE([], [], []);) 2025.09.25 21:11:06.810439 [ 2707 ] {99710493-db6e-4c52-b287-c9a4577ee071} TCPHandler: Code: 48. DB::Exception: Java hash is not combineable for multiple arguments: In scope SELECT javaHashUTF16LE([], [], []). (NOT_IMPLEMENTED) 2025.09.25 21:11:07.048540 [ 2707 ] {82db9a04-5fc9-42a5-ab96-46adaf602016} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function joinGet, expected a const string.: In scope SELECT joinGet([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT joinGet([], [], []);) 2025.09.25 21:11:07.054885 [ 2707 ] {82db9a04-5fc9-42a5-ab96-46adaf602016} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function joinGet, expected a const string.: In scope SELECT joinGet([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:07.094823 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:07.143467 [ 2707 ] {2a6ad17a-1f30-4f03-b006-1f830862c7db} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function joinGet, expected a const string.: In scope SELECT joinGetOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT joinGetOrNull([], [], []);) 2025.09.25 21:11:07.151572 [ 2707 ] {2a6ad17a-1f30-4f03-b006-1f830862c7db} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function joinGet, expected a const string.: In scope SELECT joinGetOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:07.165807 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:07.257794 [ 2707 ] {06b74a77-ca43-4e66-8220-8161ab79cb3c} executeQuery: Code: 43. DB::Exception: Function JSONMergePatch requires string arguments: In scope SELECT jsonMergePatch([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT jsonMergePatch([], [], []);) 2025.09.25 21:11:07.279123 [ 2707 ] {06b74a77-ca43-4e66-8220-8161ab79cb3c} TCPHandler: Code: 43. DB::Exception: Function JSONMergePatch requires string arguments: In scope SELECT jsonMergePatch([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:07.386555 [ 2707 ] {2b3c399a-c401-46f0-a592-f92cb48d5805} executeQuery: Code: 42. DB::Exception: Number of arguments for function jumpConsistentHash doesn't match: passed 3, should be 2: In scope SELECT jumpConsistentHash([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT jumpConsistentHash([], [], []);) 2025.09.25 21:11:07.396222 [ 2707 ] {2b3c399a-c401-46f0-a592-f92cb48d5805} TCPHandler: Code: 42. DB::Exception: Number of arguments for function jumpConsistentHash doesn't match: passed 3, should be 2: In scope SELECT jumpConsistentHash([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:07.658001 [ 2707 ] {0673876b-9083-45f6-865f-23dc43d49a0f} executeQuery: Code: 42. DB::Exception: Aggregate function kolmogorovSmirnovTest requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT kolmogorovSmirnovTest([], [], []);) 2025.09.25 21:11:07.664446 [ 2707 ] {0673876b-9083-45f6-865f-23dc43d49a0f} TCPHandler: Code: 42. DB::Exception: Aggregate function kolmogorovSmirnovTest requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:07.754097 [ 2707 ] {22b4b5a4-dbe5-45d2-8b49-c6ca5d0ac041} executeQuery: Code: 42. DB::Exception: Number of arguments for function kostikConsistentHash doesn't match: passed 3, should be 2: In scope SELECT kostikConsistentHash([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT kostikConsistentHash([], [], []);) 2025.09.25 21:11:07.767158 [ 2707 ] {22b4b5a4-dbe5-45d2-8b49-c6ca5d0ac041} TCPHandler: Code: 42. DB::Exception: Number of arguments for function kostikConsistentHash doesn't match: passed 3, should be 2: In scope SELECT kostikConsistentHash([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:08.258928 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:08.278412 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:08.422898 [ 2707 ] {002480d6-51da-4b97-9f1a-ae8231169a56} executeQuery: Code: 42. DB::Exception: Aggregate function kurtPop requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT kurtPop([], [], []);) 2025.09.25 21:11:08.429756 [ 2707 ] {002480d6-51da-4b97-9f1a-ae8231169a56} TCPHandler: Code: 42. DB::Exception: Aggregate function kurtPop requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:08.495212 [ 2707 ] {e49e36ec-638d-47d6-a35a-02d6ed5dcf78} executeQuery: Code: 42. DB::Exception: Aggregate function kurtSamp requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT kurtSamp([], [], []);) 2025.09.25 21:11:08.500605 [ 2707 ] {e49e36ec-638d-47d6-a35a-02d6ed5dcf78} TCPHandler: Code: 42. DB::Exception: Aggregate function kurtSamp requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:08.581247 [ 2707 ] {5d5ca347-46e2-4191-81be-169ffe3bbe01} executeQuery: Code: 36. DB::Exception: Offset must be an integer, 'Array(Nothing)' given. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT lagInFrame([], [], []);) 2025.09.25 21:11:08.587208 [ 2707 ] {5d5ca347-46e2-4191-81be-169ffe3bbe01} TCPHandler: Code: 36. DB::Exception: Offset must be an integer, 'Array(Nothing)' given. (BAD_ARGUMENTS) 2025.09.25 21:11:08.664736 [ 2707 ] {fdb35657-c12b-4668-8115-13d7bbce9c14} executeQuery: Code: 42. DB::Exception: Aggregate function largestTriangleThreeBuckets requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT largestTriangleThreeBuckets([], [], []);) 2025.09.25 21:11:08.679327 [ 2707 ] {fdb35657-c12b-4668-8115-13d7bbce9c14} TCPHandler: Code: 42. DB::Exception: Aggregate function largestTriangleThreeBuckets requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:08.734952 [ 2707 ] {fab185bb-aa38-401d-9862-2033c8d762ab} executeQuery: Code: 42. DB::Exception: Aggregate function anyLast requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT last_value([], [], []);) 2025.09.25 21:11:08.739807 [ 2707 ] {fab185bb-aa38-401d-9862-2033c8d762ab} TCPHandler: Code: 42. DB::Exception: Aggregate function anyLast requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:08.795819 [ 2707 ] {2f8fe816-8344-4dd5-a203-0330df27ae33} executeQuery: Code: 42. DB::Exception: Aggregate function anyLast_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT last_value_respect_nulls([], [], []);) 2025.09.25 21:11:08.801451 [ 2707 ] {2f8fe816-8344-4dd5-a203-0330df27ae33} TCPHandler: Code: 42. DB::Exception: Aggregate function anyLast_respect_nulls requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:08.933510 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:08.973366 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:08.973545 [ 2707 ] {13961b11-1c21-44fa-8fd9-4894c94a98b1} executeQuery: Code: 42. DB::Exception: Number of arguments for function lower doesn't match: passed 3, should be 1: In scope SELECT lcase([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT lcase([], [], []);) 2025.09.25 21:11:08.980484 [ 2707 ] {13961b11-1c21-44fa-8fd9-4894c94a98b1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function lower doesn't match: passed 3, should be 1: In scope SELECT lcase([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:09.147899 [ 2707 ] {b32a6fb9-2c6b-412c-945a-2daa5c517af0} executeQuery: Code: 42. DB::Exception: Number of arguments for function lcm doesn't match: passed 3, should be 2: In scope SELECT lcm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT lcm([], [], []);) 2025.09.25 21:11:09.159794 [ 2707 ] {b32a6fb9-2c6b-412c-945a-2daa5c517af0} TCPHandler: Code: 42. DB::Exception: Number of arguments for function lcm doesn't match: passed 3, should be 2: In scope SELECT lcm([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:09.238820 [ 2707 ] {25434550-1128-47dd-9881-b07f5bdf43c6} executeQuery: Code: 36. DB::Exception: Offset must be an integer, 'Array(Nothing)' given. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT leadInFrame([], [], []);) 2025.09.25 21:11:09.249162 [ 2707 ] {25434550-1128-47dd-9881-b07f5bdf43c6} TCPHandler: Code: 36. DB::Exception: Offset must be an integer, 'Array(Nothing)' given. (BAD_ARGUMENTS) 2025.09.25 21:11:09.459493 [ 2707 ] {cf02308a-4c75-433f-bfba-26bf762e06e8} executeQuery: Code: 42. DB::Exception: Number of arguments for function left doesn't match: passed 3, should be 2: In scope SELECT left([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT left([], [], []);) 2025.09.25 21:11:09.471539 [ 2707 ] {cf02308a-4c75-433f-bfba-26bf762e06e8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function left doesn't match: passed 3, should be 2: In scope SELECT left([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:09.594338 [ 2707 ] {5b6b7f97-631c-483e-9500-5784a7ca045c} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of the first argument of function leftPad, should be string: In scope SELECT leftPad([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT leftPad([], [], []);) 2025.09.25 21:11:09.600262 [ 2707 ] {5b6b7f97-631c-483e-9500-5784a7ca045c} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of the first argument of function leftPad, should be string: In scope SELECT leftPad([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:09.659578 [ 2707 ] {9a34355f-34ec-42b7-9ba8-90ca0859d1c1} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of the first argument of function leftPadUTF8, should be string: In scope SELECT leftPadUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT leftPadUTF8([], [], []);) 2025.09.25 21:11:09.736098 [ 2707 ] {9a34355f-34ec-42b7-9ba8-90ca0859d1c1} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of the first argument of function leftPadUTF8, should be string: In scope SELECT leftPadUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:09.859472 [ 2707 ] {59ecda51-fa02-449e-adb1-c4192782438d} executeQuery: Code: 42. DB::Exception: Number of arguments for function leftUTF8 doesn't match: passed 3, should be 2: In scope SELECT leftUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT leftUTF8([], [], []);) 2025.09.25 21:11:09.866233 [ 2707 ] {59ecda51-fa02-449e-adb1-c4192782438d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function leftUTF8 doesn't match: passed 3, should be 2: In scope SELECT leftUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:09.924370 [ 2707 ] {0f6d5998-b3f0-4c2b-96c9-751b99e852b9} executeQuery: Code: 344. DB::Exception: Natural language processing function 'lemmatize' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT lemmatize([], [], []);) 2025.09.25 21:11:09.932846 [ 2707 ] {0f6d5998-b3f0-4c2b-96c9-751b99e852b9} TCPHandler: Code: 344. DB::Exception: Natural language processing function 'lemmatize' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) 2025.09.25 21:11:10.031163 [ 2707 ] {ecfdc6e9-9e0d-4d35-9921-c22009b1a291} executeQuery: Code: 42. DB::Exception: Number of arguments for function length doesn't match: passed 3, should be 1: In scope SELECT length([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT length([], [], []);) 2025.09.25 21:11:10.036149 [ 2707 ] {ecfdc6e9-9e0d-4d35-9921-c22009b1a291} TCPHandler: Code: 42. DB::Exception: Number of arguments for function length doesn't match: passed 3, should be 1: In scope SELECT length([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:10.092110 [ 2707 ] {50e56786-e3ae-4164-bf1d-a73756e94fd9} executeQuery: Code: 42. DB::Exception: Number of arguments for function lengthUTF8 doesn't match: passed 3, should be 1: In scope SELECT lengthUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT lengthUTF8([], [], []);) 2025.09.25 21:11:10.096415 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:10.097983 [ 2707 ] {50e56786-e3ae-4164-bf1d-a73756e94fd9} TCPHandler: Code: 42. DB::Exception: Number of arguments for function lengthUTF8 doesn't match: passed 3, should be 1: In scope SELECT lengthUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:10.154261 [ 2707 ] {1e05aa27-c9e8-4a64-818f-ba672e893ae6} executeQuery: Code: 42. DB::Exception: Number of arguments for function less doesn't match: passed 3, should be 2: In scope SELECT less([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT less([], [], []);) 2025.09.25 21:11:10.159856 [ 2707 ] {1e05aa27-c9e8-4a64-818f-ba672e893ae6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function less doesn't match: passed 3, should be 2: In scope SELECT less([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:10.166523 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:10.233625 [ 2707 ] {01ef8847-94b6-4a8d-86b2-c1301a3fed22} executeQuery: Code: 42. DB::Exception: Number of arguments for function lessOrEquals doesn't match: passed 3, should be 2: In scope SELECT lessOrEquals([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT lessOrEquals([], [], []);) 2025.09.25 21:11:10.240114 [ 2707 ] {01ef8847-94b6-4a8d-86b2-c1301a3fed22} TCPHandler: Code: 42. DB::Exception: Number of arguments for function lessOrEquals doesn't match: passed 3, should be 2: In scope SELECT lessOrEquals([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:10.301591 [ 2707 ] {3c1c395d-c36f-417d-ae79-2531fd01a7a3} executeQuery: Code: 42. DB::Exception: Number of arguments for function editDistance doesn't match: passed 3, should be 2: In scope SELECT levenshteinDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT levenshteinDistance([], [], []);) 2025.09.25 21:11:10.307826 [ 2707 ] {3c1c395d-c36f-417d-ae79-2531fd01a7a3} TCPHandler: Code: 42. DB::Exception: Number of arguments for function editDistance doesn't match: passed 3, should be 2: In scope SELECT levenshteinDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:10.375034 [ 2707 ] {9c16157e-b78a-42ae-8d44-6795d25dfda5} executeQuery: Code: 42. DB::Exception: Number of arguments for function editDistanceUTF8 doesn't match: passed 3, should be 2: In scope SELECT levenshteinDistanceUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT levenshteinDistanceUTF8([], [], []);) 2025.09.25 21:11:10.478585 [ 2707 ] {9c16157e-b78a-42ae-8d44-6795d25dfda5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function editDistanceUTF8 doesn't match: passed 3, should be 2: In scope SELECT levenshteinDistanceUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:10.599834 [ 2707 ] {4822f245-7a63-4c1a-8c46-554167b3452c} executeQuery: Code: 42. DB::Exception: Number of arguments for function lgamma doesn't match: passed 3, should be 1: In scope SELECT lgamma([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT lgamma([], [], []);) 2025.09.25 21:11:10.605130 [ 2707 ] {4822f245-7a63-4c1a-8c46-554167b3452c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function lgamma doesn't match: passed 3, should be 1: In scope SELECT lgamma([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:10.814509 [ 2707 ] {b11f00c3-7aaa-4480-adcb-1c4651a324fd} executeQuery: Code: 42. DB::Exception: Number of arguments for function like doesn't match: passed 3, should be 2: In scope SELECT like([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT like([], [], []);) 2025.09.25 21:11:10.823637 [ 2707 ] {b11f00c3-7aaa-4480-adcb-1c4651a324fd} TCPHandler: Code: 42. DB::Exception: Number of arguments for function like doesn't match: passed 3, should be 2: In scope SELECT like([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:10.897232 [ 2707 ] {9235f562-db73-481e-84c8-9e12a8171c63} executeQuery: Code: 42. DB::Exception: Number of arguments for function log doesn't match: passed 3, should be 1: In scope SELECT ln([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ln([], [], []);) 2025.09.25 21:11:10.900949 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:10.905834 [ 2707 ] {9235f562-db73-481e-84c8-9e12a8171c63} TCPHandler: Code: 42. DB::Exception: Number of arguments for function log doesn't match: passed 3, should be 1: In scope SELECT ln([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:10.929837 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:10.972545 [ 2707 ] {f40565ce-2384-482c-a146-c5b48815960e} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function locate: In scope SELECT locate([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT locate([], [], []);) 2025.09.25 21:11:10.981622 [ 2707 ] {f40565ce-2384-482c-a146-c5b48815960e} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function locate: In scope SELECT locate([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:11.163821 [ 2707 ] {5b2a1920-5e5c-45a9-9a07-494f79db9ba3} executeQuery: Code: 42. DB::Exception: Number of arguments for function log doesn't match: passed 3, should be 1: In scope SELECT log([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT log([], [], []);) 2025.09.25 21:11:11.171833 [ 2707 ] {5b2a1920-5e5c-45a9-9a07-494f79db9ba3} TCPHandler: Code: 42. DB::Exception: Number of arguments for function log doesn't match: passed 3, should be 1: In scope SELECT log([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:11.338545 [ 2707 ] {8ea4390f-498e-4645-8703-322f100a4235} executeQuery: Code: 42. DB::Exception: Number of arguments for function log10 doesn't match: passed 3, should be 1: In scope SELECT log10([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT log10([], [], []);) 2025.09.25 21:11:11.343390 [ 2707 ] {8ea4390f-498e-4645-8703-322f100a4235} TCPHandler: Code: 42. DB::Exception: Number of arguments for function log10 doesn't match: passed 3, should be 1: In scope SELECT log10([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:11.489700 [ 2707 ] {19f7eed0-528c-4428-8092-0f796039b978} executeQuery: Code: 42. DB::Exception: Number of arguments for function log1p doesn't match: passed 3, should be 1: In scope SELECT log1p([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT log1p([], [], []);) 2025.09.25 21:11:11.500597 [ 2707 ] {19f7eed0-528c-4428-8092-0f796039b978} TCPHandler: Code: 42. DB::Exception: Number of arguments for function log1p doesn't match: passed 3, should be 1: In scope SELECT log1p([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:11.555336 [ 2707 ] {4b28a6a0-517a-499d-92e6-89e1314eba10} executeQuery: Code: 42. DB::Exception: Number of arguments for function log2 doesn't match: passed 3, should be 1: In scope SELECT log2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT log2([], [], []);) 2025.09.25 21:11:11.560047 [ 2707 ] {4b28a6a0-517a-499d-92e6-89e1314eba10} TCPHandler: Code: 42. DB::Exception: Number of arguments for function log2 doesn't match: passed 3, should be 1: In scope SELECT log2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:11.616144 [ 2707 ] {409f5de0-1439-4af8-92e6-90fd45b4253a} executeQuery: Code: 42. DB::Exception: Number of arguments for function logTrace doesn't match: passed 3, should be 1: In scope SELECT logTrace([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT logTrace([], [], []);) 2025.09.25 21:11:11.621249 [ 2707 ] {409f5de0-1439-4af8-92e6-90fd45b4253a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function logTrace doesn't match: passed 3, should be 1: In scope SELECT logTrace([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:11.688420 [ 2707 ] {81e53580-fcdf-4c64-a8f2-0f23661887ad} executeQuery: Code: 42. DB::Exception: Number of arguments for function lowCardinalityIndices doesn't match: passed 3, should be 1: In scope SELECT lowCardinalityIndices([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT lowCardinalityIndices([], [], []);) 2025.09.25 21:11:11.756619 [ 2707 ] {81e53580-fcdf-4c64-a8f2-0f23661887ad} TCPHandler: Code: 42. DB::Exception: Number of arguments for function lowCardinalityIndices doesn't match: passed 3, should be 1: In scope SELECT lowCardinalityIndices([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:11.871830 [ 2707 ] {94d87177-f97a-4140-ae44-3570ea071817} executeQuery: Code: 42. DB::Exception: Number of arguments for function lowCardinalityKeys doesn't match: passed 3, should be 1: In scope SELECT lowCardinalityKeys([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT lowCardinalityKeys([], [], []);) 2025.09.25 21:11:11.878463 [ 2707 ] {94d87177-f97a-4140-ae44-3570ea071817} TCPHandler: Code: 42. DB::Exception: Number of arguments for function lowCardinalityKeys doesn't match: passed 3, should be 1: In scope SELECT lowCardinalityKeys([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:11.894612 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:11.911067 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:11.935783 [ 2707 ] {80cf1901-bf80-4f2a-aae9-d409ec78bfbf} executeQuery: Code: 42. DB::Exception: Number of arguments for function lower doesn't match: passed 3, should be 1: In scope SELECT lower([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT lower([], [], []);) 2025.09.25 21:11:11.939238 [ 2707 ] {80cf1901-bf80-4f2a-aae9-d409ec78bfbf} TCPHandler: Code: 42. DB::Exception: Number of arguments for function lower doesn't match: passed 3, should be 1: In scope SELECT lower([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:12.053761 [ 2707 ] {afaa5a0e-5924-410d-b30d-23fcafe73cfa} executeQuery: Code: 42. DB::Exception: Number of arguments for function lowerUTF8 doesn't match: passed 3, should be 1: In scope SELECT lowerUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT lowerUTF8([], [], []);) 2025.09.25 21:11:12.066234 [ 2707 ] {afaa5a0e-5924-410d-b30d-23fcafe73cfa} TCPHandler: Code: 42. DB::Exception: Number of arguments for function lowerUTF8 doesn't match: passed 3, should be 1: In scope SELECT lowerUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:12.067958 [ 3361 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {ef397668-4864-4948-87da-1b70c589f025::all_71_76_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:12.134903 [ 2707 ] {28b74320-57ec-4822-919d-16db3796b031} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of the first argument of function leftPad, should be string: In scope SELECT lpad([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT lpad([], [], []);) 2025.09.25 21:11:12.140514 [ 2707 ] {28b74320-57ec-4822-919d-16db3796b031} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of the first argument of function leftPad, should be string: In scope SELECT lpad([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:12.196052 [ 2707 ] {f2491205-ccdc-4eef-b540-097d61954fb8} executeQuery: Code: 42. DB::Exception: Aggregate function largestTriangleThreeBuckets requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT lttb([], [], []);) 2025.09.25 21:11:12.203445 [ 2707 ] {f2491205-ccdc-4eef-b540-097d61954fb8} TCPHandler: Code: 42. DB::Exception: Aggregate function largestTriangleThreeBuckets requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:12.330979 [ 2707 ] {febb15e4-a522-4d21-96e4-330a05291b9d} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'year' to function 'makeDate'. Expected: Number, got: Array(Nothing): In scope SELECT makeDate([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT makeDate([], [], []);) 2025.09.25 21:11:12.338644 [ 2707 ] {febb15e4-a522-4d21-96e4-330a05291b9d} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'year' to function 'makeDate'. Expected: Number, got: Array(Nothing): In scope SELECT makeDate([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:12.394223 [ 2707 ] {6d8ae4a0-fe22-4808-a27f-6b6f8ce0794c} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'year' to function 'makeDate32'. Expected: Number, got: Array(Nothing): In scope SELECT makeDate32([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT makeDate32([], [], []);) 2025.09.25 21:11:12.400783 [ 2707 ] {6d8ae4a0-fe22-4808-a27f-6b6f8ce0794c} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'year' to function 'makeDate32'. Expected: Number, got: Array(Nothing): In scope SELECT makeDate32([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:12.487246 [ 2707 ] {595cda39-bdc1-498a-9bcc-b0f87d56a2e7} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'makeDateTime'. Expected 6 mandatory arguments and 1 optional argument, got 3 arguments: In scope SELECT makeDateTime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT makeDateTime([], [], []);) 2025.09.25 21:11:12.496052 [ 2707 ] {595cda39-bdc1-498a-9bcc-b0f87d56a2e7} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'makeDateTime'. Expected 6 mandatory arguments and 1 optional argument, got 3 arguments: In scope SELECT makeDateTime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:12.631458 [ 2707 ] {78387e39-5e93-4968-a91a-d7ee3cdf52ff} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'makeDateTime64'. Expected 6 mandatory arguments and 3 optional arguments, got 3 arguments: In scope SELECT makeDateTime64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT makeDateTime64([], [], []);) 2025.09.25 21:11:12.635372 [ 2707 ] {78387e39-5e93-4968-a91a-d7ee3cdf52ff} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'makeDateTime64'. Expected 6 mandatory arguments and 3 optional arguments, got 3 arguments: In scope SELECT makeDateTime64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:12.690344 [ 2707 ] {9ce63b5d-e8f5-429a-ab23-9179e8b813b4} executeQuery: Code: 42. DB::Exception: Aggregate function mannWhitneyUTest requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mannWhitneyUTest([], [], []);) 2025.09.25 21:11:12.697410 [ 2707 ] {9ce63b5d-e8f5-429a-ab23-9179e8b813b4} TCPHandler: Code: 42. DB::Exception: Aggregate function mannWhitneyUTest requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:12.810957 [ 3362 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {ef397668-4864-4948-87da-1b70c589f025::all_89_94_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:12.822542 [ 2707 ] {06697170-0462-4f96-8604-c08b897a1544} executeQuery: Code: 42. DB::Exception: Function map requires even number of arguments, but 3 given: In scope SELECT map([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT map([], [], []);) 2025.09.25 21:11:12.839555 [ 2707 ] {06697170-0462-4f96-8604-c08b897a1544} TCPHandler: Code: 42. DB::Exception: Function map requires even number of arguments, but 3 given: In scope SELECT map([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:12.872641 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:12.907254 [ 2707 ] {36878fa9-7ae2-46e2-9c91-4a70a507f5ff} executeQuery: Code: 43. DB::Exception: mapAdd only accepts maps: In scope SELECT mapAdd([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapAdd([], [], []);) 2025.09.25 21:11:12.914670 [ 2707 ] {36878fa9-7ae2-46e2-9c91-4a70a507f5ff} TCPHandler: Code: 43. DB::Exception: mapAdd only accepts maps: In scope SELECT mapAdd([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:12.986474 [ 2707 ] {d068dadf-8940-446b-b842-38c0ec6c80c1} executeQuery: Code: 43. DB::Exception: Function mapAll requires at least one argument of type Map: In scope SELECT mapAll([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapAll([], [], []);) 2025.09.25 21:11:12.986618 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:12.992928 [ 2707 ] {d068dadf-8940-446b-b842-38c0ec6c80c1} TCPHandler: Code: 43. DB::Exception: Function mapAll requires at least one argument of type Map: In scope SELECT mapAll([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:13.056345 [ 2707 ] {3fad39a2-90a1-4d9c-bcd5-225f3e55133d} executeQuery: Code: 43. DB::Exception: Function mapApply requires at least one argument of type Map: In scope SELECT mapApply([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapApply([], [], []);) 2025.09.25 21:11:13.063102 [ 2707 ] {3fad39a2-90a1-4d9c-bcd5-225f3e55133d} TCPHandler: Code: 43. DB::Exception: Function mapApply requires at least one argument of type Map: In scope SELECT mapApply([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:13.121348 [ 2707 ] {ca6a48bb-61b6-48f4-9b69-9588114ca0dd} executeQuery: Code: 43. DB::Exception: Function mapConcat requires at least one argument of type Map: In scope SELECT mapConcat([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapConcat([], [], []);) 2025.09.25 21:11:13.126566 [ 2707 ] {ca6a48bb-61b6-48f4-9b69-9588114ca0dd} TCPHandler: Code: 43. DB::Exception: Function mapConcat requires at least one argument of type Map: In scope SELECT mapConcat([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:13.193548 [ 2707 ] {87cde19a-d496-4ca8-964d-f68c4f24c051} executeQuery: Code: 42. DB::Exception: Number of arguments for function mapContains doesn't match: passed 3, should be 2: In scope SELECT mapContains([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapContains([], [], []);) 2025.09.25 21:11:13.205416 [ 2707 ] {87cde19a-d496-4ca8-964d-f68c4f24c051} TCPHandler: Code: 42. DB::Exception: Number of arguments for function mapContains doesn't match: passed 3, should be 2: In scope SELECT mapContains([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:13.261496 [ 2707 ] {2f0fc1ed-80a0-4c90-a4c3-329276ad9374} executeQuery: Code: 42. DB::Exception: Number of arguments for function mapContainsKeyLike doesn't match: passed 3, should be 2: In scope SELECT mapContainsKeyLike([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapContainsKeyLike([], [], []);) 2025.09.25 21:11:13.266054 [ 2707 ] {2f0fc1ed-80a0-4c90-a4c3-329276ad9374} TCPHandler: Code: 42. DB::Exception: Number of arguments for function mapContainsKeyLike doesn't match: passed 3, should be 2: In scope SELECT mapContainsKeyLike([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:13.322011 [ 2707 ] {12b0af33-4aae-4089-aafb-043524ec5e55} executeQuery: Code: 43. DB::Exception: Function mapExists requires at least one argument of type Map: In scope SELECT mapExists([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapExists([], [], []);) 2025.09.25 21:11:13.328196 [ 2707 ] {12b0af33-4aae-4089-aafb-043524ec5e55} TCPHandler: Code: 43. DB::Exception: Function mapExists requires at least one argument of type Map: In scope SELECT mapExists([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:13.373893 [ 2707 ] {a2b70cdb-dfff-4e2e-a41e-7bfb90387470} executeQuery: Code: 42. DB::Exception: Number of arguments for function mapExtractKeyLike doesn't match: passed 3, should be 2: In scope SELECT mapExtractKeyLike([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapExtractKeyLike([], [], []);) 2025.09.25 21:11:13.378424 [ 2707 ] {a2b70cdb-dfff-4e2e-a41e-7bfb90387470} TCPHandler: Code: 42. DB::Exception: Number of arguments for function mapExtractKeyLike doesn't match: passed 3, should be 2: In scope SELECT mapExtractKeyLike([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:13.422274 [ 2707 ] {1e937ad4-f4d1-4bbc-9c12-b9cbf63af091} executeQuery: Code: 43. DB::Exception: Function mapFilter requires at least one argument of type Map: In scope SELECT mapFilter([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapFilter([], [], []);) 2025.09.25 21:11:13.457770 [ 2707 ] {1e937ad4-f4d1-4bbc-9c12-b9cbf63af091} TCPHandler: Code: 43. DB::Exception: Function mapFilter requires at least one argument of type Map: In scope SELECT mapFilter([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:13.491907 [ 2707 ] {08196795-7388-40f0-a8cc-e10202ce5114} executeQuery: Code: 42. DB::Exception: Number of arguments for function mapFromArrays doesn't match: passed 3, should be 2: In scope SELECT mapFromArrays([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapFromArrays([], [], []);) 2025.09.25 21:11:13.495918 [ 2707 ] {08196795-7388-40f0-a8cc-e10202ce5114} TCPHandler: Code: 42. DB::Exception: Number of arguments for function mapFromArrays doesn't match: passed 3, should be 2: In scope SELECT mapFromArrays([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:13.542000 [ 2707 ] {bdcfdf84-71df-41b7-a496-25ccbe0edfb0} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument data_column. Must be String.: In scope SELECT mapFromString([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapFromString([], [], []);) 2025.09.25 21:11:13.556056 [ 2707 ] {bdcfdf84-71df-41b7-a496-25ccbe0edfb0} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument data_column. Must be String.: In scope SELECT mapFromString([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:13.677629 [ 2707 ] {d2d96d72-6397-45f2-a855-ac50a6f6c802} executeQuery: Code: 42. DB::Exception: Number of arguments for function mapKeys doesn't match: passed 3, should be 1: In scope SELECT mapKeys([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapKeys([], [], []);) 2025.09.25 21:11:13.687376 [ 2707 ] {d2d96d72-6397-45f2-a855-ac50a6f6c802} TCPHandler: Code: 42. DB::Exception: Number of arguments for function mapKeys doesn't match: passed 3, should be 1: In scope SELECT mapKeys([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:13.804614 [ 2707 ] {3ee68b81-76a9-42e6-b251-0c23def3a0a2} executeQuery: Code: 43. DB::Exception: Function mapPartialReverseSort requires at least one argument of type Map: In scope SELECT mapPartialReverseSort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapPartialReverseSort([], [], []);) 2025.09.25 21:11:13.815822 [ 2707 ] {3ee68b81-76a9-42e6-b251-0c23def3a0a2} TCPHandler: Code: 43. DB::Exception: Function mapPartialReverseSort requires at least one argument of type Map: In scope SELECT mapPartialReverseSort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:13.920269 [ 2707 ] {07b9b0ff-03d5-4e82-af79-a6bf199ffa56} executeQuery: Code: 43. DB::Exception: Function mapPartialSort requires at least one argument of type Map: In scope SELECT mapPartialSort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapPartialSort([], [], []);) 2025.09.25 21:11:13.926755 [ 2707 ] {07b9b0ff-03d5-4e82-af79-a6bf199ffa56} TCPHandler: Code: 43. DB::Exception: Function mapPartialSort requires at least one argument of type Map: In scope SELECT mapPartialSort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:14.026137 [ 2707 ] {42225aa1-b254-4a1f-8c88-83a44fc2a980} executeQuery: Code: 43. DB::Exception: Function mapPopulateSeries key argument should be of signed or unsigned integer type. Actual type Nothing: In scope SELECT mapPopulateSeries([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapPopulateSeries([], [], []);) 2025.09.25 21:11:14.035714 [ 2707 ] {42225aa1-b254-4a1f-8c88-83a44fc2a980} TCPHandler: Code: 43. DB::Exception: Function mapPopulateSeries key argument should be of signed or unsigned integer type. Actual type Nothing: In scope SELECT mapPopulateSeries([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:14.070516 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:14.133737 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:14.143517 [ 2707 ] {540da603-e149-4223-a224-11a278182cf0} executeQuery: Code: 43. DB::Exception: Function mapReverseSort requires at least one argument of type Map: In scope SELECT mapReverseSort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapReverseSort([], [], []);) 2025.09.25 21:11:14.153097 [ 2707 ] {540da603-e149-4223-a224-11a278182cf0} TCPHandler: Code: 43. DB::Exception: Function mapReverseSort requires at least one argument of type Map: In scope SELECT mapReverseSort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:14.249168 [ 2707 ] {a86c459d-96ec-4491-afd2-5331bfe9ca0b} executeQuery: Code: 43. DB::Exception: Function mapSort requires at least one argument of type Map: In scope SELECT mapSort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapSort([], [], []);) 2025.09.25 21:11:14.255528 [ 2707 ] {a86c459d-96ec-4491-afd2-5331bfe9ca0b} TCPHandler: Code: 43. DB::Exception: Function mapSort requires at least one argument of type Map: In scope SELECT mapSort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:14.319995 [ 2707 ] {69c64bb5-debc-4791-ac95-886096821058} executeQuery: Code: 43. DB::Exception: mapSubtract only accepts maps: In scope SELECT mapSubtract([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapSubtract([], [], []);) 2025.09.25 21:11:14.325752 [ 2707 ] {69c64bb5-debc-4791-ac95-886096821058} TCPHandler: Code: 43. DB::Exception: mapSubtract only accepts maps: In scope SELECT mapSubtract([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:14.388503 [ 2707 ] {1ac82ac7-4e4f-40a4-8833-92519ed7b728} executeQuery: Code: 42. DB::Exception: Number of arguments for function mapUpdate doesn't match: passed 3, should be 2: In scope SELECT mapUpdate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapUpdate([], [], []);) 2025.09.25 21:11:14.395965 [ 2707 ] {1ac82ac7-4e4f-40a4-8833-92519ed7b728} TCPHandler: Code: 42. DB::Exception: Number of arguments for function mapUpdate doesn't match: passed 3, should be 2: In scope SELECT mapUpdate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:14.474650 [ 2707 ] {9ef02781-f9db-44a2-886f-916b4a398bf9} executeQuery: Code: 42. DB::Exception: Number of arguments for function mapValues doesn't match: passed 3, should be 1: In scope SELECT mapValues([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mapValues([], [], []);) 2025.09.25 21:11:14.480242 [ 2707 ] {9ef02781-f9db-44a2-886f-916b4a398bf9} TCPHandler: Code: 42. DB::Exception: Number of arguments for function mapValues doesn't match: passed 3, should be 1: In scope SELECT mapValues([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:14.538144 [ 2707 ] {bcd5d18d-8d58-45f1-ad42-2ec0b4a3c0b3} executeQuery: Code: 42. DB::Exception: Number of arguments for function match doesn't match: passed 3, should be 2: In scope SELECT match([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT match([], [], []);) 2025.09.25 21:11:14.543602 [ 2707 ] {bcd5d18d-8d58-45f1-ad42-2ec0b4a3c0b3} TCPHandler: Code: 42. DB::Exception: Number of arguments for function match doesn't match: passed 3, should be 2: In scope SELECT match([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:14.601691 [ 7168 ] {7e63b357-2fb7-4311-9197-054e3aeb366d} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=724). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:40598) (comment: 01913_quantile_deterministic.sh) (in query: SELECT medianDeterministic(oid, oid) FROM d), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:11:14.616190 [ 7168 ] {7e63b357-2fb7-4311-9197-054e3aeb366d} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=724). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:11:14.673426 [ 2707 ] {2485dcad-5a53-4f0f-b1ce-af3eb4be2f55} executeQuery: Code: 42. DB::Exception: Number of arguments for function materialize doesn't match: passed 3, should be 1: In scope SELECT materialize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT materialize([], [], []);) 2025.09.25 21:11:14.684597 [ 2707 ] {2485dcad-5a53-4f0f-b1ce-af3eb4be2f55} TCPHandler: Code: 42. DB::Exception: Number of arguments for function materialize doesn't match: passed 3, should be 1: In scope SELECT materialize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:14.776657 [ 2707 ] {b4d3f5e8-2d57-4dcd-8794-9b8751c6c3c1} executeQuery: Code: 42. DB::Exception: Aggregate function max requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT max([], [], []);) 2025.09.25 21:11:14.784149 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:14.795190 [ 2707 ] {b4d3f5e8-2d57-4dcd-8794-9b8751c6c3c1} TCPHandler: Code: 42. DB::Exception: Aggregate function max requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:14.842270 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:14.946189 [ 2707 ] {a355ee5d-bb03-496a-b82c-32036bd06763} executeQuery: Code: 42. DB::Exception: Number of arguments for function max2 doesn't match: passed 3, should be 2: In scope SELECT max2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT max2([], [], []);) 2025.09.25 21:11:14.974490 [ 2707 ] {a355ee5d-bb03-496a-b82c-32036bd06763} TCPHandler: Code: 42. DB::Exception: Number of arguments for function max2 doesn't match: passed 3, should be 2: In scope SELECT max2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:15.065189 [ 2707 ] {5f410fab-f777-4a00-81d1-ab2a421116a7} executeQuery: Code: 42. DB::Exception: Aggregate function maxIntersections requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT maxIntersections([], [], []);) 2025.09.25 21:11:15.069685 [ 2707 ] {5f410fab-f777-4a00-81d1-ab2a421116a7} TCPHandler: Code: 42. DB::Exception: Aggregate function maxIntersections requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:15.265903 [ 2707 ] {aa8d474c-9a0f-4491-aea6-39544889eb16} executeQuery: Code: 42. DB::Exception: Aggregate function maxIntersectionsPosition requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT maxIntersectionsPosition([], [], []);) 2025.09.25 21:11:15.274201 [ 2707 ] {aa8d474c-9a0f-4491-aea6-39544889eb16} TCPHandler: Code: 42. DB::Exception: Aggregate function maxIntersectionsPosition requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:15.568387 [ 2707 ] {8d6665e8-939c-4c3b-b6da-e8d1aeafc8ef} executeQuery: Code: 42. DB::Exception: Aggregate function meanZTest requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT meanZTest([], [], []);) 2025.09.25 21:11:15.572435 [ 2707 ] {8d6665e8-939c-4c3b-b6da-e8d1aeafc8ef} TCPHandler: Code: 42. DB::Exception: Aggregate function meanZTest requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:15.642269 [ 2707 ] {be16ae7e-9358-4f7c-a2c8-4e11751b3353} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantile. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT median([], [], []);) 2025.09.25 21:11:15.647119 [ 2707 ] {be16ae7e-9358-4f7c-a2c8-4e11751b3353} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantile. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:15.745783 [ 2707 ] {71fa9b0c-c85e-47c7-9a17-d8fc34b816bf} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileBFloat16. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT medianBFloat16([], [], []);) 2025.09.25 21:11:15.755882 [ 2707 ] {71fa9b0c-c85e-47c7-9a17-d8fc34b816bf} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileBFloat16. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:15.808735 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:15.810801 [ 2707 ] {f3ca9a60-4010-4708-8017-490aac608d4b} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileBFloat16Weighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT medianBFloat16Weighted([], [], []);) 2025.09.25 21:11:15.815391 [ 2707 ] {f3ca9a60-4010-4708-8017-490aac608d4b} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileBFloat16Weighted. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:15.822256 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:15.857385 [ 2707 ] {0a65baa3-3b77-453c-8af6-2ee370032505} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileDD. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT medianDD([], [], []);) 2025.09.25 21:11:15.863930 [ 2707 ] {0a65baa3-3b77-453c-8af6-2ee370032505} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileDD. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:15.922225 [ 2707 ] {ed08c55f-00fc-4c93-9477-30e3d931f6c6} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileDeterministic. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT medianDeterministic([], [], []);) 2025.09.25 21:11:15.932022 [ 2707 ] {ed08c55f-00fc-4c93-9477-30e3d931f6c6} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileDeterministic. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:15.990885 [ 2707 ] {0b06e2bf-fb8e-4761-a659-2714a55cd5c0} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExact. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT medianExact([], [], []);) 2025.09.25 21:11:15.994005 [ 2707 ] {0b06e2bf-fb8e-4761-a659-2714a55cd5c0} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExact. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:16.047049 [ 2707 ] {680d5589-b2aa-46d8-a601-a6281a05fce4} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExactHigh. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT medianExactHigh([], [], []);) 2025.09.25 21:11:16.055459 [ 2707 ] {680d5589-b2aa-46d8-a601-a6281a05fce4} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExactHigh. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:16.227265 [ 2707 ] {feb6e115-fc13-4788-9253-b64ad7d75145} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExactLow. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT medianExactLow([], [], []);) 2025.09.25 21:11:16.235911 [ 2707 ] {feb6e115-fc13-4788-9253-b64ad7d75145} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExactLow. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:16.315719 [ 2707 ] {8009baab-0cca-429a-adea-c29cc1bb3401} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExactWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT medianExactWeighted([], [], []);) 2025.09.25 21:11:16.325444 [ 2707 ] {8009baab-0cca-429a-adea-c29cc1bb3401} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExactWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:16.369754 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:16.419495 [ 2707 ] {ea8a4082-c38b-464b-a772-d7e90a2c2be8} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileGK. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT medianGK([], [], []);) 2025.09.25 21:11:16.439824 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:16.451415 [ 2707 ] {ea8a4082-c38b-464b-a772-d7e90a2c2be8} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileGK. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:16.620876 [ 2707 ] {f5a347e6-4e36-4eae-89db-e7b122fe1d61} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileInterpolatedWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT medianInterpolatedWeighted([], [], []);) 2025.09.25 21:11:16.638759 [ 2707 ] {f5a347e6-4e36-4eae-89db-e7b122fe1d61} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileInterpolatedWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:16.766537 [ 2707 ] {44e059c6-49c1-45a6-838e-b017739a4699} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileTDigest. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT medianTDigest([], [], []);) 2025.09.25 21:11:16.774945 [ 2707 ] {44e059c6-49c1-45a6-838e-b017739a4699} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileTDigest. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:16.845578 [ 2707 ] {d7cf9fa7-7cc9-48a8-81bc-3b0e27f57815} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileTDigestWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT medianTDigestWeighted([], [], []);) 2025.09.25 21:11:16.850441 [ 2707 ] {d7cf9fa7-7cc9-48a8-81bc-3b0e27f57815} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileTDigestWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:16.918705 [ 2707 ] {b2039b76-f44b-4dac-8f85-d51f75dfccfb} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileTiming. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT medianTiming([], [], []);) 2025.09.25 21:11:16.927109 [ 2707 ] {b2039b76-f44b-4dac-8f85-d51f75dfccfb} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileTiming. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:17.028053 [ 2707 ] {99070c0c-e81f-497d-bb41-b37f3333797a} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileTimingWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT medianTimingWeighted([], [], []);) 2025.09.25 21:11:17.039199 [ 2707 ] {99070c0c-e81f-497d-bb41-b37f3333797a} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileTimingWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:17.051081 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:17.089474 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:17.268273 [ 2707 ] {36380a19-924f-4cff-99ba-daa8fab58262} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function substring, expected String, FixedString or Enum: In scope SELECT mid([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mid([], [], []);) 2025.09.25 21:11:17.282486 [ 2707 ] {36380a19-924f-4cff-99ba-daa8fab58262} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function substring, expected String, FixedString or Enum: In scope SELECT mid([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:17.393049 [ 2707 ] {f9582082-dbd2-4dc7-ba6c-4a5bc4c231e1} executeQuery: Code: 42. DB::Exception: Aggregate function min requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT min([], [], []);) 2025.09.25 21:11:17.399108 [ 2707 ] {f9582082-dbd2-4dc7-ba6c-4a5bc4c231e1} TCPHandler: Code: 42. DB::Exception: Aggregate function min requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:17.552861 [ 2707 ] {54cf42b0-7013-4564-91e7-3221a2431b57} executeQuery: Code: 42. DB::Exception: Number of arguments for function min2 doesn't match: passed 3, should be 2: In scope SELECT min2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT min2([], [], []);) 2025.09.25 21:11:17.562341 [ 2707 ] {54cf42b0-7013-4564-91e7-3221a2431b57} TCPHandler: Code: 42. DB::Exception: Number of arguments for function min2 doesn't match: passed 3, should be 2: In scope SELECT min2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:17.679666 [ 3682 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:17.978969 [ 2707 ] {9db715b9-f306-4c83-a23b-2946fba16508} executeQuery: Code: 42. DB::Exception: Number of arguments for function minSampleSizeContinuous doesn't match: passed 3, should be 5: In scope SELECT minSampleSizeContinous([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT minSampleSizeContinous([], [], []);) 2025.09.25 21:11:17.990394 [ 2707 ] {9db715b9-f306-4c83-a23b-2946fba16508} TCPHandler: Code: 42. DB::Exception: Number of arguments for function minSampleSizeContinuous doesn't match: passed 3, should be 5: In scope SELECT minSampleSizeContinous([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:18.114086 [ 2707 ] {aa52acf2-a7d3-4c29-8165-8c28c86b12fc} executeQuery: Code: 42. DB::Exception: Number of arguments for function minSampleSizeContinuous doesn't match: passed 3, should be 5: In scope SELECT minSampleSizeContinuous([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT minSampleSizeContinuous([], [], []);) 2025.09.25 21:11:18.128306 [ 2707 ] {aa52acf2-a7d3-4c29-8165-8c28c86b12fc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function minSampleSizeContinuous doesn't match: passed 3, should be 5: In scope SELECT minSampleSizeContinuous([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:18.151113 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:18.187949 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:18.253218 [ 2707 ] {71bab2d6-5621-4ec8-a52e-af98728c6a73} executeQuery: Code: 42. DB::Exception: Number of arguments for function minSampleSizeConversion doesn't match: passed 3, should be 4: In scope SELECT minSampleSizeConversion([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT minSampleSizeConversion([], [], []);) 2025.09.25 21:11:18.261926 [ 2707 ] {71bab2d6-5621-4ec8-a52e-af98728c6a73} TCPHandler: Code: 42. DB::Exception: Number of arguments for function minSampleSizeConversion doesn't match: passed 3, should be 4: In scope SELECT minSampleSizeConversion([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:18.394460 [ 2707 ] {5f2e9702-f192-43c2-8d8f-105cfc762a04} executeQuery: Code: 42. DB::Exception: Number of arguments for function minus doesn't match: passed 3, should be 2: In scope SELECT minus([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT minus([], [], []);) 2025.09.25 21:11:18.419831 [ 2707 ] {5f2e9702-f192-43c2-8d8f-105cfc762a04} TCPHandler: Code: 42. DB::Exception: Number of arguments for function minus doesn't match: passed 3, should be 2: In scope SELECT minus([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:18.494265 [ 2707 ] {8b3484dc-52cd-4805-8f28-d669587657e1} executeQuery: Code: 42. DB::Exception: Number of arguments for function byteHammingDistance doesn't match: passed 3, should be 2: In scope SELECT mismatches([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mismatches([], [], []);) 2025.09.25 21:11:18.501854 [ 2707 ] {8b3484dc-52cd-4805-8f28-d669587657e1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function byteHammingDistance doesn't match: passed 3, should be 2: In scope SELECT mismatches([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:18.576918 [ 2707 ] {a527224b-8502-4d09-adf8-50337064e2dd} executeQuery: Code: 42. DB::Exception: Number of arguments for function modulo doesn't match: passed 3, should be 2: In scope SELECT mod([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mod([], [], []);) 2025.09.25 21:11:18.584884 [ 2707 ] {a527224b-8502-4d09-adf8-50337064e2dd} TCPHandler: Code: 42. DB::Exception: Number of arguments for function modulo doesn't match: passed 3, should be 2: In scope SELECT mod([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:18.664521 [ 2707 ] {cb346a32-c1e3-4500-b604-35104c761012} executeQuery: Code: 42. DB::Exception: Number of arguments for function modulo doesn't match: passed 3, should be 2: In scope SELECT modulo([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT modulo([], [], []);) 2025.09.25 21:11:18.670814 [ 2707 ] {cb346a32-c1e3-4500-b604-35104c761012} TCPHandler: Code: 42. DB::Exception: Number of arguments for function modulo doesn't match: passed 3, should be 2: In scope SELECT modulo([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:18.830797 [ 2707 ] {e09caac0-41ea-4d79-8590-00aef7cb3f36} executeQuery: Code: 42. DB::Exception: Number of arguments for function moduloLegacy doesn't match: passed 3, should be 2: In scope SELECT moduloLegacy([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT moduloLegacy([], [], []);) 2025.09.25 21:11:18.838025 [ 2707 ] {e09caac0-41ea-4d79-8590-00aef7cb3f36} TCPHandler: Code: 42. DB::Exception: Number of arguments for function moduloLegacy doesn't match: passed 3, should be 2: In scope SELECT moduloLegacy([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:19.088840 [ 2707 ] {9f5e3b0f-31da-4c73-bd38-c3b4adfd2540} executeQuery: Code: 42. DB::Exception: Number of arguments for function moduloOrZero doesn't match: passed 3, should be 2: In scope SELECT moduloOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT moduloOrZero([], [], []);) 2025.09.25 21:11:19.093343 [ 2707 ] {9f5e3b0f-31da-4c73-bd38-c3b4adfd2540} TCPHandler: Code: 42. DB::Exception: Number of arguments for function moduloOrZero doesn't match: passed 3, should be 2: In scope SELECT moduloOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:19.189012 [ 2707 ] {d0f277b1-8502-4c2e-971c-51662d4b6048} executeQuery: Code: 42. DB::Exception: Number of arguments for function monthName doesn't match: passed 3, should be 1: In scope SELECT monthName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT monthName([], [], []);) 2025.09.25 21:11:19.201728 [ 2707 ] {d0f277b1-8502-4c2e-971c-51662d4b6048} TCPHandler: Code: 42. DB::Exception: Number of arguments for function monthName doesn't match: passed 3, should be 1: In scope SELECT monthName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:19.322649 [ 2707 ] {7f5a3976-54ea-41c8-a8d1-94b92eb5e1e4} executeQuery: Code: 42. DB::Exception: Number of arguments for function mortonDecode doesn't match: passed 3, should be 2: In scope SELECT mortonDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mortonDecode([], [], []);) 2025.09.25 21:11:19.346341 [ 2707 ] {7f5a3976-54ea-41c8-a8d1-94b92eb5e1e4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function mortonDecode doesn't match: passed 3, should be 2: In scope SELECT mortonDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:19.393796 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:19.455545 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:19.473464 [ 2707 ] {3c3a6675-8f77-42ac-a4e7-151f193c8f45} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for function mortonEncode, should be a native UInt: In scope SELECT mortonEncode([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT mortonEncode([], [], []);) 2025.09.25 21:11:19.485085 [ 2707 ] {3c3a6675-8f77-42ac-a4e7-151f193c8f45} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for function mortonEncode, should be a native UInt: In scope SELECT mortonEncode([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:19.581831 [ 2707 ] {694fee3a-4aca-4ddd-8ecc-0307abd03e55} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function multiFuzzyMatchAllIndices: In scope SELECT multiFuzzyMatchAllIndices([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiFuzzyMatchAllIndices([], [], []);) 2025.09.25 21:11:19.590690 [ 2707 ] {694fee3a-4aca-4ddd-8ecc-0307abd03e55} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function multiFuzzyMatchAllIndices: In scope SELECT multiFuzzyMatchAllIndices([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:19.798430 [ 2707 ] {17359e24-3e99-4076-ba1a-b9f6f20e7f74} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function multiFuzzyMatchAny: In scope SELECT multiFuzzyMatchAny([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiFuzzyMatchAny([], [], []);) 2025.09.25 21:11:19.805098 [ 2707 ] {17359e24-3e99-4076-ba1a-b9f6f20e7f74} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function multiFuzzyMatchAny: In scope SELECT multiFuzzyMatchAny([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:19.918401 [ 2707 ] {dc988dd3-04bb-412a-a31f-309dd50017ea} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function multiFuzzyMatchAnyIndex: In scope SELECT multiFuzzyMatchAnyIndex([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiFuzzyMatchAnyIndex([], [], []);) 2025.09.25 21:11:19.927174 [ 2707 ] {dc988dd3-04bb-412a-a31f-309dd50017ea} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function multiFuzzyMatchAnyIndex: In scope SELECT multiFuzzyMatchAnyIndex([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:20.080845 [ 2707 ] {d350f12d-d822-416f-adba-ed1b70c4eb27} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument (condition) of function multiIf. Must be UInt8.: In scope SELECT multiIf([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiIf([], [], []);) 2025.09.25 21:11:20.095823 [ 2707 ] {d350f12d-d822-416f-adba-ed1b70c4eb27} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument (condition) of function multiIf. Must be UInt8.: In scope SELECT multiIf([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:20.188096 [ 2707 ] {9873eb9b-b4e1-46df-9a9a-4623326530cc} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiMatchAllIndices doesn't match: passed 3, should be 2: In scope SELECT multiMatchAllIndices([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiMatchAllIndices([], [], []);) 2025.09.25 21:11:20.197866 [ 2707 ] {9873eb9b-b4e1-46df-9a9a-4623326530cc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiMatchAllIndices doesn't match: passed 3, should be 2: In scope SELECT multiMatchAllIndices([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:20.301616 [ 2707 ] {a60f80b6-55bd-41ca-ab87-024fccdcc25c} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiMatchAny doesn't match: passed 3, should be 2: In scope SELECT multiMatchAny([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiMatchAny([], [], []);) 2025.09.25 21:11:20.306898 [ 2707 ] {a60f80b6-55bd-41ca-ab87-024fccdcc25c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiMatchAny doesn't match: passed 3, should be 2: In scope SELECT multiMatchAny([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:20.372446 [ 2707 ] {82fada29-0f11-437e-ac47-46c088e31275} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiMatchAnyIndex doesn't match: passed 3, should be 2: In scope SELECT multiMatchAnyIndex([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiMatchAnyIndex([], [], []);) 2025.09.25 21:11:20.382995 [ 2707 ] {82fada29-0f11-437e-ac47-46c088e31275} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiMatchAnyIndex doesn't match: passed 3, should be 2: In scope SELECT multiMatchAnyIndex([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:20.469210 [ 2707 ] {86f6aae3-5c4b-4431-b950-9889a048cf8f} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchAllPositions doesn't match: passed 3, should be 2: In scope SELECT multiSearchAllPositions([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiSearchAllPositions([], [], []);) 2025.09.25 21:11:20.485052 [ 2707 ] {86f6aae3-5c4b-4431-b950-9889a048cf8f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchAllPositions doesn't match: passed 3, should be 2: In scope SELECT multiSearchAllPositions([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:20.555317 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:20.600514 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:20.612375 [ 2707 ] {e201822a-7d29-4dae-916b-e5342238ffdd} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchAllPositionsCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT multiSearchAllPositionsCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiSearchAllPositionsCaseInsensitive([], [], []);) 2025.09.25 21:11:20.618086 [ 2707 ] {e201822a-7d29-4dae-916b-e5342238ffdd} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchAllPositionsCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT multiSearchAllPositionsCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:20.728922 [ 2707 ] {88b7a49b-6408-40d0-9c2a-a45874c0816a} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchAllPositionsCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchAllPositionsCaseInsensitiveUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiSearchAllPositionsCaseInsensitiveUTF8([], [], []);) 2025.09.25 21:11:20.734450 [ 2707 ] {88b7a49b-6408-40d0-9c2a-a45874c0816a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchAllPositionsCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchAllPositionsCaseInsensitiveUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:21.008992 [ 2707 ] {fffc207b-53b4-4468-95cb-b776bfc9e549} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchAllPositionsUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchAllPositionsUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiSearchAllPositionsUTF8([], [], []);) 2025.09.25 21:11:21.013778 [ 2707 ] {fffc207b-53b4-4468-95cb-b776bfc9e549} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchAllPositionsUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchAllPositionsUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:21.187521 [ 2707 ] {88d029b0-4468-4e40-a8c1-cff91ac7e443} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchAny doesn't match: passed 3, should be 2: In scope SELECT multiSearchAny([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiSearchAny([], [], []);) 2025.09.25 21:11:21.196216 [ 2707 ] {88d029b0-4468-4e40-a8c1-cff91ac7e443} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchAny doesn't match: passed 3, should be 2: In scope SELECT multiSearchAny([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:21.368850 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:21.380470 [ 2707 ] {0a628bf8-71e3-4a73-99ba-2867149ee674} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchAnyCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT multiSearchAnyCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiSearchAnyCaseInsensitive([], [], []);) 2025.09.25 21:11:21.391915 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:21.393623 [ 2707 ] {0a628bf8-71e3-4a73-99ba-2867149ee674} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchAnyCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT multiSearchAnyCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:21.498633 [ 2707 ] {dc52eb24-5d8e-4bbc-854a-0423cfc0aef5} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchAnyCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchAnyCaseInsensitiveUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiSearchAnyCaseInsensitiveUTF8([], [], []);) 2025.09.25 21:11:21.505867 [ 2707 ] {dc52eb24-5d8e-4bbc-854a-0423cfc0aef5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchAnyCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchAnyCaseInsensitiveUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:21.621570 [ 2707 ] {473e81c2-3849-4940-8e57-7156d00ba434} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchAnyUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchAnyUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiSearchAnyUTF8([], [], []);) 2025.09.25 21:11:21.635785 [ 2707 ] {473e81c2-3849-4940-8e57-7156d00ba434} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchAnyUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchAnyUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:21.710162 [ 2707 ] {fd667e97-43fa-4a89-8345-9a33f7c4a91f} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstIndex doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstIndex([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiSearchFirstIndex([], [], []);) 2025.09.25 21:11:21.722985 [ 2707 ] {fd667e97-43fa-4a89-8345-9a33f7c4a91f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstIndex doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstIndex([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:21.806692 [ 2707 ] {6bb35a55-6de6-4f3d-bd76-4fd75ceabd14} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstIndexCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstIndexCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiSearchFirstIndexCaseInsensitive([], [], []);) 2025.09.25 21:11:21.815966 [ 2707 ] {6bb35a55-6de6-4f3d-bd76-4fd75ceabd14} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstIndexCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstIndexCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:21.901062 [ 2707 ] {495e2846-3457-4b56-a4a9-8ac808ea51bc} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstIndexCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstIndexCaseInsensitiveUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiSearchFirstIndexCaseInsensitiveUTF8([], [], []);) 2025.09.25 21:11:21.920625 [ 2707 ] {495e2846-3457-4b56-a4a9-8ac808ea51bc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstIndexCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstIndexCaseInsensitiveUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:22.069894 [ 2707 ] {859d056d-0ac8-48e1-8e12-b764d9183cd4} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstIndexUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstIndexUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiSearchFirstIndexUTF8([], [], []);) 2025.09.25 21:11:22.076565 [ 2707 ] {859d056d-0ac8-48e1-8e12-b764d9183cd4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstIndexUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstIndexUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:22.152276 [ 2707 ] {b91f6b5b-16ba-46a2-9bd3-a5ffed10c15d} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstPosition doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstPosition([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiSearchFirstPosition([], [], []);) 2025.09.25 21:11:22.160192 [ 2707 ] {b91f6b5b-16ba-46a2-9bd3-a5ffed10c15d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstPosition doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstPosition([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:22.265354 [ 2707 ] {c6ff1aa7-0b35-4188-918a-af335de302e2} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstPositionCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstPositionCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiSearchFirstPositionCaseInsensitive([], [], []);) 2025.09.25 21:11:22.272720 [ 2707 ] {c6ff1aa7-0b35-4188-918a-af335de302e2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstPositionCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstPositionCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:22.342379 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:22.373712 [ 2707 ] {83cb7354-379d-4273-b2fe-d0f1bdf7e5fa} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstPositionCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstPositionCaseInsensitiveUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiSearchFirstPositionCaseInsensitiveUTF8([], [], []);) 2025.09.25 21:11:22.385544 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:22.402253 [ 2707 ] {83cb7354-379d-4273-b2fe-d0f1bdf7e5fa} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstPositionCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstPositionCaseInsensitiveUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:22.491711 [ 2707 ] {7f18829f-f7e9-45af-a170-57ba1c71173b} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstPositionUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstPositionUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiSearchFirstPositionUTF8([], [], []);) 2025.09.25 21:11:22.498635 [ 2707 ] {7f18829f-f7e9-45af-a170-57ba1c71173b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiSearchFirstPositionUTF8 doesn't match: passed 3, should be 2: In scope SELECT multiSearchFirstPositionUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:22.594011 [ 2707 ] {5c702099-eb1d-430c-8352-3c63010dcd90} executeQuery: Code: 42. DB::Exception: Number of arguments for function multiply doesn't match: passed 3, should be 2: In scope SELECT multiply([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiply([], [], []);) 2025.09.25 21:11:22.671259 [ 2707 ] {5c702099-eb1d-430c-8352-3c63010dcd90} TCPHandler: Code: 42. DB::Exception: Number of arguments for function multiply doesn't match: passed 3, should be 2: In scope SELECT multiply([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:22.740912 [ 2707 ] {56abfe5f-968d-466e-aebf-e653a9de5c23} executeQuery: Code: 43. DB::Exception: Arguments for multiplyDecimal function must be Decimal: In scope SELECT multiplyDecimal([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT multiplyDecimal([], [], []);) 2025.09.25 21:11:22.748263 [ 2707 ] {56abfe5f-968d-466e-aebf-e653a9de5c23} TCPHandler: Code: 43. DB::Exception: Arguments for multiplyDecimal function must be Decimal: In scope SELECT multiplyDecimal([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:22.826666 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:22.897691 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:23.379624 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:23.412978 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:23.594289 [ 2707 ] {9b8972c0-6ab4-400a-95f9-0766b0f2a619} executeQuery: Code: 42. DB::Exception: Number of arguments for function negate doesn't match: passed 3, should be 1: In scope SELECT negate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT negate([], [], []);) 2025.09.25 21:11:23.611723 [ 2707 ] {9b8972c0-6ab4-400a-95f9-0766b0f2a619} TCPHandler: Code: 42. DB::Exception: Number of arguments for function negate doesn't match: passed 3, should be 1: In scope SELECT negate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:23.771406 [ 2707 ] {d7f15647-d797-4ff1-bf26-3e0909afcc44} executeQuery: Code: 721. DB::Exception: Function neighbor is deprecated since its usage is error-prone (see docs).Please use proper window function or set `allow_deprecated_error_prone_window_functions` setting to enable it. (DEPRECATED_FUNCTION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT neighbor([], [], []);) 2025.09.25 21:11:23.776296 [ 2707 ] {d7f15647-d797-4ff1-bf26-3e0909afcc44} TCPHandler: Code: 721. DB::Exception: Function neighbor is deprecated since its usage is error-prone (see docs).Please use proper window function or set `allow_deprecated_error_prone_window_functions` setting to enable it. (DEPRECATED_FUNCTION) 2025.09.25 21:11:23.881694 [ 2707 ] {9854a8d2-614b-4a5a-bfa8-049273212fa3} executeQuery: Code: 36. DB::Exception: First argument for function nested must be constant column with array of strings: In scope SELECT nested([], [], []). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT nested([], [], []);) 2025.09.25 21:11:23.888033 [ 2707 ] {9854a8d2-614b-4a5a-bfa8-049273212fa3} TCPHandler: Code: 36. DB::Exception: First argument for function nested must be constant column with array of strings: In scope SELECT nested([], [], []). (BAD_ARGUMENTS) 2025.09.25 21:11:23.988331 [ 2707 ] {c3bb4535-8420-4980-a9f2-fa11e9cfb614} executeQuery: Code: 42. DB::Exception: Number of arguments for function netloc doesn't match: passed 3, should be 1: In scope SELECT netloc([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT netloc([], [], []);) 2025.09.25 21:11:23.995282 [ 2707 ] {c3bb4535-8420-4980-a9f2-fa11e9cfb614} TCPHandler: Code: 42. DB::Exception: Number of arguments for function netloc doesn't match: passed 3, should be 1: In scope SELECT netloc([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:24.092394 [ 2707 ] {8ea36c4a-7ca6-4cbf-95b4-f53f739f762b} executeQuery: Code: 42. DB::Exception: Number of arguments for function ngramDistance doesn't match: passed 3, should be 2: In scope SELECT ngramDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramDistance([], [], []);) 2025.09.25 21:11:24.104560 [ 2707 ] {8ea36c4a-7ca6-4cbf-95b4-f53f739f762b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngramDistance doesn't match: passed 3, should be 2: In scope SELECT ngramDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:24.209802 [ 2707 ] {477c56d2-cacf-44ec-bb84-82eba1b410c5} executeQuery: Code: 42. DB::Exception: Number of arguments for function ngramDistanceCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT ngramDistanceCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramDistanceCaseInsensitive([], [], []);) 2025.09.25 21:11:24.216687 [ 2707 ] {477c56d2-cacf-44ec-bb84-82eba1b410c5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngramDistanceCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT ngramDistanceCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:24.344216 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:24.345159 [ 2707 ] {f567f81c-def8-4cf0-b076-bf048a8e3e52} executeQuery: Code: 42. DB::Exception: Number of arguments for function ngramDistanceCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT ngramDistanceCaseInsensitiveUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramDistanceCaseInsensitiveUTF8([], [], []);) 2025.09.25 21:11:24.353579 [ 2707 ] {f567f81c-def8-4cf0-b076-bf048a8e3e52} TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngramDistanceCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT ngramDistanceCaseInsensitiveUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:24.388954 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:24.504872 [ 2707 ] {37452244-3e5c-407f-afd2-a327d94329fe} executeQuery: Code: 42. DB::Exception: Number of arguments for function ngramDistanceUTF8 doesn't match: passed 3, should be 2: In scope SELECT ngramDistanceUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramDistanceUTF8([], [], []);) 2025.09.25 21:11:24.518623 [ 2707 ] {37452244-3e5c-407f-afd2-a327d94329fe} TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngramDistanceUTF8 doesn't match: passed 3, should be 2: In scope SELECT ngramDistanceUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:24.698927 [ 2707 ] {db2e10c3-9967-4cd6-afcb-74fa3702e74b} executeQuery: Code: 43. DB::Exception: First argument of function ngramMinHash must be String, got Array(Nothing): In scope SELECT ngramMinHash([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramMinHash([], [], []);) 2025.09.25 21:11:24.705309 [ 2707 ] {db2e10c3-9967-4cd6-afcb-74fa3702e74b} TCPHandler: Code: 43. DB::Exception: First argument of function ngramMinHash must be String, got Array(Nothing): In scope SELECT ngramMinHash([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:24.805719 [ 2707 ] {0f819758-fc7c-4e6c-8e01-fc505dc7a2c8} executeQuery: Code: 43. DB::Exception: First argument of function ngramMinHashArg must be String, got Array(Nothing): In scope SELECT ngramMinHashArg([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramMinHashArg([], [], []);) 2025.09.25 21:11:24.817325 [ 2707 ] {0f819758-fc7c-4e6c-8e01-fc505dc7a2c8} TCPHandler: Code: 43. DB::Exception: First argument of function ngramMinHashArg must be String, got Array(Nothing): In scope SELECT ngramMinHashArg([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:24.902681 [ 2707 ] {d9a25163-499f-461d-a204-57ec3c09c792} executeQuery: Code: 43. DB::Exception: First argument of function ngramMinHashArgCaseInsensitive must be String, got Array(Nothing): In scope SELECT ngramMinHashArgCaseInsensitive([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramMinHashArgCaseInsensitive([], [], []);) 2025.09.25 21:11:24.910586 [ 2707 ] {d9a25163-499f-461d-a204-57ec3c09c792} TCPHandler: Code: 43. DB::Exception: First argument of function ngramMinHashArgCaseInsensitive must be String, got Array(Nothing): In scope SELECT ngramMinHashArgCaseInsensitive([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:25.067510 [ 2707 ] {8c3ee37d-4126-4f4c-a9c4-8c17bbb7caee} executeQuery: Code: 43. DB::Exception: First argument of function ngramMinHashArgCaseInsensitiveUTF8 must be String, got Array(Nothing): In scope SELECT ngramMinHashArgCaseInsensitiveUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramMinHashArgCaseInsensitiveUTF8([], [], []);) 2025.09.25 21:11:25.075140 [ 2707 ] {8c3ee37d-4126-4f4c-a9c4-8c17bbb7caee} TCPHandler: Code: 43. DB::Exception: First argument of function ngramMinHashArgCaseInsensitiveUTF8 must be String, got Array(Nothing): In scope SELECT ngramMinHashArgCaseInsensitiveUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:25.194082 [ 2707 ] {34d3d965-35dc-46ed-bd69-6c1d9246d771} executeQuery: Code: 43. DB::Exception: First argument of function ngramMinHashArgUTF8 must be String, got Array(Nothing): In scope SELECT ngramMinHashArgUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramMinHashArgUTF8([], [], []);) 2025.09.25 21:11:25.215903 [ 2707 ] {34d3d965-35dc-46ed-bd69-6c1d9246d771} TCPHandler: Code: 43. DB::Exception: First argument of function ngramMinHashArgUTF8 must be String, got Array(Nothing): In scope SELECT ngramMinHashArgUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:25.399799 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:25.463571 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:25.522702 [ 2707 ] {5b0ee40d-96de-4e9b-aaf2-e0022501e815} executeQuery: Code: 43. DB::Exception: First argument of function ngramMinHashCaseInsensitive must be String, got Array(Nothing): In scope SELECT ngramMinHashCaseInsensitive([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramMinHashCaseInsensitive([], [], []);) 2025.09.25 21:11:25.527483 [ 2707 ] {5b0ee40d-96de-4e9b-aaf2-e0022501e815} TCPHandler: Code: 43. DB::Exception: First argument of function ngramMinHashCaseInsensitive must be String, got Array(Nothing): In scope SELECT ngramMinHashCaseInsensitive([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:25.707412 [ 2707 ] {3b78d7d0-a280-417c-b33a-13504856cef0} executeQuery: Code: 43. DB::Exception: First argument of function ngramMinHashCaseInsensitiveUTF8 must be String, got Array(Nothing): In scope SELECT ngramMinHashCaseInsensitiveUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramMinHashCaseInsensitiveUTF8([], [], []);) 2025.09.25 21:11:25.714454 [ 2707 ] {3b78d7d0-a280-417c-b33a-13504856cef0} TCPHandler: Code: 43. DB::Exception: First argument of function ngramMinHashCaseInsensitiveUTF8 must be String, got Array(Nothing): In scope SELECT ngramMinHashCaseInsensitiveUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:25.787646 [ 2707 ] {39e7c8a9-66d7-456d-a5f7-b76d4684199f} executeQuery: Code: 43. DB::Exception: First argument of function ngramMinHashUTF8 must be String, got Array(Nothing): In scope SELECT ngramMinHashUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramMinHashUTF8([], [], []);) 2025.09.25 21:11:25.793101 [ 2707 ] {39e7c8a9-66d7-456d-a5f7-b76d4684199f} TCPHandler: Code: 43. DB::Exception: First argument of function ngramMinHashUTF8 must be String, got Array(Nothing): In scope SELECT ngramMinHashUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:25.809417 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:25.835836 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:25.849883 [ 2707 ] {728c2182-bd12-4718-b446-080b404600ec} executeQuery: Code: 42. DB::Exception: Number of arguments for function ngramSearch doesn't match: passed 3, should be 2: In scope SELECT ngramSearch([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramSearch([], [], []);) 2025.09.25 21:11:25.857162 [ 2707 ] {728c2182-bd12-4718-b446-080b404600ec} TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngramSearch doesn't match: passed 3, should be 2: In scope SELECT ngramSearch([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:26.019069 [ 2707 ] {11bc2fac-7718-440b-ae8b-287626ec952e} executeQuery: Code: 42. DB::Exception: Number of arguments for function ngramSearchCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT ngramSearchCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramSearchCaseInsensitive([], [], []);) 2025.09.25 21:11:26.035078 [ 2707 ] {11bc2fac-7718-440b-ae8b-287626ec952e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngramSearchCaseInsensitive doesn't match: passed 3, should be 2: In scope SELECT ngramSearchCaseInsensitive([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:26.142845 [ 2707 ] {e16f092d-b291-44b4-adb2-491a98a04191} executeQuery: Code: 42. DB::Exception: Number of arguments for function ngramSearchCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT ngramSearchCaseInsensitiveUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramSearchCaseInsensitiveUTF8([], [], []);) 2025.09.25 21:11:26.160407 [ 2707 ] {e16f092d-b291-44b4-adb2-491a98a04191} TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngramSearchCaseInsensitiveUTF8 doesn't match: passed 3, should be 2: In scope SELECT ngramSearchCaseInsensitiveUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:26.292956 [ 2707 ] {548c630d-c598-4517-b0d9-de748c5b3494} executeQuery: Code: 42. DB::Exception: Number of arguments for function ngramSearchUTF8 doesn't match: passed 3, should be 2: In scope SELECT ngramSearchUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramSearchUTF8([], [], []);) 2025.09.25 21:11:26.297595 [ 2707 ] {548c630d-c598-4517-b0d9-de748c5b3494} TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngramSearchUTF8 doesn't match: passed 3, should be 2: In scope SELECT ngramSearchUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:26.420530 [ 2707 ] {f2c0dfe4-4e39-4481-89a4-4c96e2e501ec} executeQuery: Code: 43. DB::Exception: First argument of function ngramSimHash must be String, got Array(Nothing): In scope SELECT ngramSimHash([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramSimHash([], [], []);) 2025.09.25 21:11:26.430233 [ 2707 ] {f2c0dfe4-4e39-4481-89a4-4c96e2e501ec} TCPHandler: Code: 43. DB::Exception: First argument of function ngramSimHash must be String, got Array(Nothing): In scope SELECT ngramSimHash([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:26.503106 [ 2707 ] {ae1b1c02-0e24-4d90-b56d-bacbaf5d5e48} executeQuery: Code: 43. DB::Exception: First argument of function ngramSimHashCaseInsensitive must be String, got Array(Nothing): In scope SELECT ngramSimHashCaseInsensitive([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramSimHashCaseInsensitive([], [], []);) 2025.09.25 21:11:26.511034 [ 2707 ] {ae1b1c02-0e24-4d90-b56d-bacbaf5d5e48} TCPHandler: Code: 43. DB::Exception: First argument of function ngramSimHashCaseInsensitive must be String, got Array(Nothing): In scope SELECT ngramSimHashCaseInsensitive([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:26.613091 [ 2707 ] {bbe08578-af9f-4fad-9548-f73b1b8fa9b7} executeQuery: Code: 43. DB::Exception: First argument of function ngramSimHashCaseInsensitiveUTF8 must be String, got Array(Nothing): In scope SELECT ngramSimHashCaseInsensitiveUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramSimHashCaseInsensitiveUTF8([], [], []);) 2025.09.25 21:11:26.623665 [ 2707 ] {bbe08578-af9f-4fad-9548-f73b1b8fa9b7} TCPHandler: Code: 43. DB::Exception: First argument of function ngramSimHashCaseInsensitiveUTF8 must be String, got Array(Nothing): In scope SELECT ngramSimHashCaseInsensitiveUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:26.715364 [ 2707 ] {1265c7dc-6cb2-4a73-9c01-d8a98b0643ad} executeQuery: Code: 43. DB::Exception: First argument of function ngramSimHashUTF8 must be String, got Array(Nothing): In scope SELECT ngramSimHashUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngramSimHashUTF8([], [], []);) 2025.09.25 21:11:26.721639 [ 2707 ] {1265c7dc-6cb2-4a73-9c01-d8a98b0643ad} TCPHandler: Code: 43. DB::Exception: First argument of function ngramSimHashUTF8 must be String, got Array(Nothing): In scope SELECT ngramSimHashUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:26.777780 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:26.794191 [ 2707 ] {3d6a3f7d-b801-4907-995a-e69ea44e6ce4} executeQuery: Code: 42. DB::Exception: Number of arguments for function ngrams doesn't match: passed 3, should be 2: In scope SELECT ngrams([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ngrams([], [], []);) 2025.09.25 21:11:26.806786 [ 2707 ] {3d6a3f7d-b801-4907-995a-e69ea44e6ce4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function ngrams doesn't match: passed 3, should be 2: In scope SELECT ngrams([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:26.856518 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:26.883755 [ 2707 ] {7aa070d8-0c4a-433d-8fe7-2ecd64c15ca0} executeQuery: Code: 36. DB::Exception: Argument 0 must be a number, 'Array(Nothing)' given. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT nonNegativeDerivative([], [], []);) 2025.09.25 21:11:26.896786 [ 2707 ] {7aa070d8-0c4a-433d-8fe7-2ecd64c15ca0} TCPHandler: Code: 36. DB::Exception: Argument 0 must be a number, 'Array(Nothing)' given. (BAD_ARGUMENTS) 2025.09.25 21:11:27.007970 [ 2707 ] {419efcff-6019-4ac0-aa06-521c09c47f24} executeQuery: Code: 42. DB::Exception: Number of arguments for function L1Norm doesn't match: passed 3, should be 1: In scope SELECT normL1([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normL1([], [], []);) 2025.09.25 21:11:27.015887 [ 2707 ] {419efcff-6019-4ac0-aa06-521c09c47f24} TCPHandler: Code: 42. DB::Exception: Number of arguments for function L1Norm doesn't match: passed 3, should be 1: In scope SELECT normL1([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:27.130981 [ 2707 ] {50ad88fe-d0ee-4b51-813e-ca30882cc782} executeQuery: Code: 42. DB::Exception: Number of arguments for function L2Norm doesn't match: passed 3, should be 1: In scope SELECT normL2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normL2([], [], []);) 2025.09.25 21:11:27.136272 [ 2707 ] {50ad88fe-d0ee-4b51-813e-ca30882cc782} TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2Norm doesn't match: passed 3, should be 1: In scope SELECT normL2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:27.216199 [ 2707 ] {b173a431-cee9-45cf-a96c-5c99c318359a} executeQuery: Code: 42. DB::Exception: Number of arguments for function L2SquaredNorm doesn't match: passed 3, should be 1: In scope SELECT normL2Squared([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normL2Squared([], [], []);) 2025.09.25 21:11:27.227015 [ 2707 ] {b173a431-cee9-45cf-a96c-5c99c318359a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2SquaredNorm doesn't match: passed 3, should be 1: In scope SELECT normL2Squared([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:27.304193 [ 2707 ] {6615d8e7-1992-4983-bef7-0b192c41df71} executeQuery: Code: 42. DB::Exception: Number of arguments for function LinfNorm doesn't match: passed 3, should be 1: In scope SELECT normLinf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normLinf([], [], []);) 2025.09.25 21:11:27.310520 [ 2707 ] {6615d8e7-1992-4983-bef7-0b192c41df71} TCPHandler: Code: 42. DB::Exception: Number of arguments for function LinfNorm doesn't match: passed 3, should be 1: In scope SELECT normLinf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:27.400923 [ 2707 ] {37ff7319-2e9e-4bc9-b1be-bf61e0a0610c} executeQuery: Code: 42. DB::Exception: Number of arguments for function LpNorm doesn't match: passed 3, should be 2: In scope SELECT normLp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normLp([], [], []);) 2025.09.25 21:11:27.409765 [ 2707 ] {37ff7319-2e9e-4bc9-b1be-bf61e0a0610c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function LpNorm doesn't match: passed 3, should be 2: In scope SELECT normLp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:27.506187 [ 2707 ] {771a8473-c6a3-40fc-bcc8-9a9d1891e428} executeQuery: Code: 42. DB::Exception: Number of arguments for function L1Normalize doesn't match: passed 3, should be 1: In scope SELECT normalizeL1([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normalizeL1([], [], []);) 2025.09.25 21:11:27.511955 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:27.518002 [ 2707 ] {771a8473-c6a3-40fc-bcc8-9a9d1891e428} TCPHandler: Code: 42. DB::Exception: Number of arguments for function L1Normalize doesn't match: passed 3, should be 1: In scope SELECT normalizeL1([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:27.547631 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:27.606401 [ 2707 ] {0b764044-e76d-4626-a01f-41d293c190e8} executeQuery: Code: 42. DB::Exception: Number of arguments for function L2Normalize doesn't match: passed 3, should be 1: In scope SELECT normalizeL2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normalizeL2([], [], []);) 2025.09.25 21:11:27.616614 [ 2707 ] {0b764044-e76d-4626-a01f-41d293c190e8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function L2Normalize doesn't match: passed 3, should be 1: In scope SELECT normalizeL2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:27.682941 [ 2707 ] {28f29a29-5972-4966-9edc-0322945d0b6a} executeQuery: Code: 42. DB::Exception: Number of arguments for function LinfNormalize doesn't match: passed 3, should be 1: In scope SELECT normalizeLinf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normalizeLinf([], [], []);) 2025.09.25 21:11:27.688776 [ 2707 ] {28f29a29-5972-4966-9edc-0322945d0b6a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function LinfNormalize doesn't match: passed 3, should be 1: In scope SELECT normalizeLinf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:27.766884 [ 2707 ] {b8d0a052-1885-4b75-a8bc-c51dee9e5d17} executeQuery: Code: 42. DB::Exception: Number of arguments for function LpNormalize doesn't match: passed 3, should be 2: In scope SELECT normalizeLp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normalizeLp([], [], []);) 2025.09.25 21:11:27.771671 [ 2707 ] {b8d0a052-1885-4b75-a8bc-c51dee9e5d17} TCPHandler: Code: 42. DB::Exception: Number of arguments for function LpNormalize doesn't match: passed 3, should be 2: In scope SELECT normalizeLp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:27.911320 [ 2707 ] {d22d2bc0-b26d-4271-adbe-1ade4559356b} executeQuery: Code: 42. DB::Exception: Number of arguments for function normalizeQuery doesn't match: passed 3, should be 1: In scope SELECT normalizeQuery([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normalizeQuery([], [], []);) 2025.09.25 21:11:27.919244 [ 2707 ] {d22d2bc0-b26d-4271-adbe-1ade4559356b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function normalizeQuery doesn't match: passed 3, should be 1: In scope SELECT normalizeQuery([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:28.005801 [ 2707 ] {52f58134-a243-46ca-8792-017ce2419e17} executeQuery: Code: 42. DB::Exception: Number of arguments for function normalizeQueryKeepNames doesn't match: passed 3, should be 1: In scope SELECT normalizeQueryKeepNames([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normalizeQueryKeepNames([], [], []);) 2025.09.25 21:11:28.036081 [ 2707 ] {52f58134-a243-46ca-8792-017ce2419e17} TCPHandler: Code: 42. DB::Exception: Number of arguments for function normalizeQueryKeepNames doesn't match: passed 3, should be 1: In scope SELECT normalizeQueryKeepNames([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:28.134027 [ 2707 ] {67a30ebe-9ae2-442f-be90-78a58918e518} executeQuery: Code: 42. DB::Exception: Number of arguments for function normalizeUTF8NFC doesn't match: passed 3, should be 1: In scope SELECT normalizeUTF8NFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normalizeUTF8NFC([], [], []);) 2025.09.25 21:11:28.137651 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:28.207500 [ 2707 ] {67a30ebe-9ae2-442f-be90-78a58918e518} TCPHandler: Code: 42. DB::Exception: Number of arguments for function normalizeUTF8NFC doesn't match: passed 3, should be 1: In scope SELECT normalizeUTF8NFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:28.220491 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:28.272626 [ 2707 ] {cf162873-d028-4657-823b-2297fb0add0f} executeQuery: Code: 42. DB::Exception: Number of arguments for function normalizeUTF8NFD doesn't match: passed 3, should be 1: In scope SELECT normalizeUTF8NFD([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normalizeUTF8NFD([], [], []);) 2025.09.25 21:11:28.288085 [ 2707 ] {cf162873-d028-4657-823b-2297fb0add0f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function normalizeUTF8NFD doesn't match: passed 3, should be 1: In scope SELECT normalizeUTF8NFD([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:28.352155 [ 2707 ] {2ce2c561-dfe1-44a9-a6c5-08317a9f2688} executeQuery: Code: 42. DB::Exception: Number of arguments for function normalizeUTF8NFKC doesn't match: passed 3, should be 1: In scope SELECT normalizeUTF8NFKC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normalizeUTF8NFKC([], [], []);) 2025.09.25 21:11:28.376293 [ 2707 ] {2ce2c561-dfe1-44a9-a6c5-08317a9f2688} TCPHandler: Code: 42. DB::Exception: Number of arguments for function normalizeUTF8NFKC doesn't match: passed 3, should be 1: In scope SELECT normalizeUTF8NFKC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:28.497080 [ 2707 ] {c945c3b6-1b86-4460-a062-493a5d1437a2} executeQuery: Code: 42. DB::Exception: Number of arguments for function normalizeUTF8NFKD doesn't match: passed 3, should be 1: In scope SELECT normalizeUTF8NFKD([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normalizeUTF8NFKD([], [], []);) 2025.09.25 21:11:28.509607 [ 2707 ] {c945c3b6-1b86-4460-a062-493a5d1437a2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function normalizeUTF8NFKD doesn't match: passed 3, should be 1: In scope SELECT normalizeUTF8NFKD([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:28.571466 [ 2707 ] {34612b1b-6c80-4635-8eaf-8f04ddc3943d} executeQuery: Code: 42. DB::Exception: Number of arguments for function normalizedQueryHash doesn't match: passed 3, should be 1: In scope SELECT normalizedQueryHash([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normalizedQueryHash([], [], []);) 2025.09.25 21:11:28.577330 [ 2707 ] {34612b1b-6c80-4635-8eaf-8f04ddc3943d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function normalizedQueryHash doesn't match: passed 3, should be 1: In scope SELECT normalizedQueryHash([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:28.636817 [ 2707 ] {df4ed9ab-1614-4dfd-993c-a6c920ab7fa8} executeQuery: Code: 42. DB::Exception: Number of arguments for function normalizedQueryHashKeepNames doesn't match: passed 3, should be 1: In scope SELECT normalizedQueryHashKeepNames([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT normalizedQueryHashKeepNames([], [], []);) 2025.09.25 21:11:28.647530 [ 2707 ] {df4ed9ab-1614-4dfd-993c-a6c920ab7fa8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function normalizedQueryHashKeepNames doesn't match: passed 3, should be 1: In scope SELECT normalizedQueryHashKeepNames([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:28.704382 [ 2707 ] {926edd16-bcc6-4047-b95c-568444e170c5} executeQuery: Code: 42. DB::Exception: Number of arguments for function not doesn't match: passed 3, should be 1: In scope SELECT not([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT not([], [], []);) 2025.09.25 21:11:28.712274 [ 2707 ] {926edd16-bcc6-4047-b95c-568444e170c5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function not doesn't match: passed 3, should be 1: In scope SELECT not([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:28.802496 [ 2707 ] {4a34ece2-5519-41f9-b978-74bece941ea7} executeQuery: Code: 42. DB::Exception: Number of arguments for function notEmpty doesn't match: passed 3, should be 1: In scope SELECT notEmpty([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT notEmpty([], [], []);) 2025.09.25 21:11:28.812128 [ 2707 ] {4a34ece2-5519-41f9-b978-74bece941ea7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function notEmpty doesn't match: passed 3, should be 1: In scope SELECT notEmpty([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:28.910067 [ 2707 ] {3ff45e10-5c32-481e-bfe6-d9d91c3aa9ba} executeQuery: Code: 42. DB::Exception: Number of arguments for function notEquals doesn't match: passed 3, should be 2: In scope SELECT notEquals([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT notEquals([], [], []);) 2025.09.25 21:11:28.936840 [ 2707 ] {3ff45e10-5c32-481e-bfe6-d9d91c3aa9ba} TCPHandler: Code: 42. DB::Exception: Number of arguments for function notEquals doesn't match: passed 3, should be 2: In scope SELECT notEquals([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:29.031054 [ 2707 ] {668c00ff-5867-4525-a985-3b37c2f78eea} executeQuery: Code: 42. DB::Exception: Number of arguments for function notILike doesn't match: passed 3, should be 2: In scope SELECT notILike([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT notILike([], [], []);) 2025.09.25 21:11:29.042630 [ 2707 ] {668c00ff-5867-4525-a985-3b37c2f78eea} TCPHandler: Code: 42. DB::Exception: Number of arguments for function notILike doesn't match: passed 3, should be 2: In scope SELECT notILike([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:29.104844 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:29.116012 [ 2707 ] {abcba36d-e14a-4139-aee8-29eb5ff2b40d} executeQuery: Code: 42. DB::Exception: Function 'notIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT notIn([], [], []);) 2025.09.25 21:11:29.121630 [ 2707 ] {abcba36d-e14a-4139-aee8-29eb5ff2b40d} TCPHandler: Code: 42. DB::Exception: Function 'notIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:29.183798 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:29.220316 [ 2707 ] {08fd6b39-f0f6-4b30-8745-bbcd97525415} executeQuery: Code: 42. DB::Exception: Function 'notInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT notInIgnoreSet([], [], []);) 2025.09.25 21:11:29.225642 [ 2707 ] {08fd6b39-f0f6-4b30-8745-bbcd97525415} TCPHandler: Code: 42. DB::Exception: Function 'notInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:29.332740 [ 2707 ] {ba8791d9-b6e4-4c82-9bf4-cb93a425f7a7} executeQuery: Code: 42. DB::Exception: Number of arguments for function notLike doesn't match: passed 3, should be 2: In scope SELECT notLike([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT notLike([], [], []);) 2025.09.25 21:11:29.339678 [ 2707 ] {ba8791d9-b6e4-4c82-9bf4-cb93a425f7a7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function notLike doesn't match: passed 3, should be 2: In scope SELECT notLike([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:29.448804 [ 2707 ] {0fa401a0-7f8e-43a8-9d9a-1a17f937df0b} executeQuery: Code: 42. DB::Exception: Function 'notNullIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT notNullIn([], [], []);) 2025.09.25 21:11:29.455841 [ 2707 ] {0fa401a0-7f8e-43a8-9d9a-1a17f937df0b} TCPHandler: Code: 42. DB::Exception: Function 'notNullIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:29.494717 [ 3351 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {68caad89-531d-40c0-87c3-4b3574cf49ff::202509_1_30_6}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:29.581116 [ 2707 ] {ac2e077a-dd86-4fdf-9b3d-c011e565b647} executeQuery: Code: 42. DB::Exception: Function 'notNullInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT notNullInIgnoreSet([], [], []);) 2025.09.25 21:11:29.591595 [ 2707 ] {ac2e077a-dd86-4fdf-9b3d-c011e565b647} TCPHandler: Code: 42. DB::Exception: Function 'notNullInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:29.637889 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:29.783064 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:30.286014 [ 2707 ] {a6d55771-99a6-4d03-bb4c-02b1ac790e82} executeQuery: Code: 42. DB::Exception: Arguments size of function now should be 0 or 1: In scope SELECT now([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT now([], [], []);) 2025.09.25 21:11:30.293699 [ 2707 ] {a6d55771-99a6-4d03-bb4c-02b1ac790e82} TCPHandler: Code: 42. DB::Exception: Arguments size of function now should be 0 or 1: In scope SELECT now([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:30.390583 [ 2707 ] {1568e671-ea75-429f-b95d-c12688d226fb} executeQuery: Code: 34. DB::Exception: Arguments size of function now64 should be 0, or 1, or 2: In scope SELECT now64([], [], []). (TOO_MANY_ARGUMENTS_FOR_FUNCTION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT now64([], [], []);) 2025.09.25 21:11:30.396671 [ 2707 ] {1568e671-ea75-429f-b95d-c12688d226fb} TCPHandler: Code: 34. DB::Exception: Arguments size of function now64 should be 0, or 1, or 2: In scope SELECT now64([], [], []). (TOO_MANY_ARGUMENTS_FOR_FUNCTION) 2025.09.25 21:11:30.443055 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:30.467061 [ 2707 ] {b02c6223-8ea5-4ddc-bcc3-2280a1d9669c} executeQuery: Code: 34. DB::Exception: Arguments size of function nowInBlock should be 0 or 1: In scope SELECT nowInBlock([], [], []). (TOO_MANY_ARGUMENTS_FOR_FUNCTION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT nowInBlock([], [], []);) 2025.09.25 21:11:30.468431 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:30.472624 [ 2707 ] {b02c6223-8ea5-4ddc-bcc3-2280a1d9669c} TCPHandler: Code: 34. DB::Exception: Arguments size of function nowInBlock should be 0 or 1: In scope SELECT nowInBlock([], [], []). (TOO_MANY_ARGUMENTS_FOR_FUNCTION) 2025.09.25 21:11:30.690489 [ 2707 ] {ff4feb60-8585-4eeb-9895-d53f10651cf4} executeQuery: Code: 42. DB::Exception: Function nth_value takes exactly two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT nth_value([], [], []);) 2025.09.25 21:11:30.696856 [ 2707 ] {ff4feb60-8585-4eeb-9895-d53f10651cf4} TCPHandler: Code: 42. DB::Exception: Function nth_value takes exactly two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:30.849157 [ 2707 ] {3cefa4b2-53ab-4df8-a713-8efe33d8ca57} executeQuery: Code: 42. DB::Exception: Function ntile takes exactly one argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ntile([], [], []);) 2025.09.25 21:11:30.863534 [ 2707 ] {3cefa4b2-53ab-4df8-a713-8efe33d8ca57} TCPHandler: Code: 42. DB::Exception: Function ntile takes exactly one argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:30.956509 [ 2707 ] {5c4a16a5-f218-4bef-bd90-71f9122e1cf5} executeQuery: Code: 42. DB::Exception: Number of arguments for function nullIf doesn't match: passed 3, should be 2: In scope SELECT nullIf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT nullIf([], [], []);) 2025.09.25 21:11:30.961800 [ 2707 ] {5c4a16a5-f218-4bef-bd90-71f9122e1cf5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function nullIf doesn't match: passed 3, should be 2: In scope SELECT nullIf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:31.048389 [ 2707 ] {7a09b617-e1de-4235-a7b7-e8d8ed202aa9} executeQuery: Code: 42. DB::Exception: Function 'nullIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT nullIn([], [], []);) 2025.09.25 21:11:31.058952 [ 2707 ] {7a09b617-e1de-4235-a7b7-e8d8ed202aa9} TCPHandler: Code: 42. DB::Exception: Function 'nullIn' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:31.135968 [ 2707 ] {6eb6c065-5146-4de1-9e72-88c50efd3aa8} executeQuery: Code: 42. DB::Exception: Function 'nullInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT nullInIgnoreSet([], [], []);) 2025.09.25 21:11:31.149133 [ 2707 ] {6eb6c065-5146-4de1-9e72-88c50efd3aa8} TCPHandler: Code: 42. DB::Exception: Function 'nullInIgnoreSet' expects 2 arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:31.273339 [ 2707 ] {7f9a0dfa-d857-4b0f-95fd-e4613683df50} executeQuery: Code: 43. DB::Exception: Illegal type (Array(Nothing)) of 1 argument of function or: In scope SELECT [] OR [] OR []. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT or([], [], []);) 2025.09.25 21:11:31.281597 [ 2707 ] {7f9a0dfa-d857-4b0f-95fd-e4613683df50} TCPHandler: Code: 43. DB::Exception: Illegal type (Array(Nothing)) of 1 argument of function or: In scope SELECT [] OR [] OR []. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:31.369431 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:31.384768 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:31.441017 [ 2707 ] {50daac55-21c4-4a96-b5ab-058a5404331f} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'time' to function 'parseDateTime'. Expected: String, got: Array(Nothing): In scope SELECT parseDateTime([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTime([], [], []);) 2025.09.25 21:11:31.446917 [ 2707 ] {50daac55-21c4-4a96-b5ab-058a5404331f} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'time' to function 'parseDateTime'. Expected: String, got: Array(Nothing): In scope SELECT parseDateTime([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:31.612461 [ 2707 ] {45ab45bb-f2bf-494f-965c-ba56b201ad8b} executeQuery: Code: 42. DB::Exception: Number of arguments for function parseDateTime32BestEffort doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT parseDateTime32BestEffort([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTime32BestEffort([], [], []);) 2025.09.25 21:11:31.619926 [ 2707 ] {45ab45bb-f2bf-494f-965c-ba56b201ad8b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function parseDateTime32BestEffort doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT parseDateTime32BestEffort([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:31.711518 [ 2707 ] {477f9c1f-c58d-4140-aabd-36a7b93b2b32} executeQuery: Code: 42. DB::Exception: Number of arguments for function parseDateTime32BestEffortOrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT parseDateTime32BestEffortOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTime32BestEffortOrNull([], [], []);) 2025.09.25 21:11:31.723897 [ 2707 ] {477f9c1f-c58d-4140-aabd-36a7b93b2b32} TCPHandler: Code: 42. DB::Exception: Number of arguments for function parseDateTime32BestEffortOrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT parseDateTime32BestEffortOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:31.856635 [ 2707 ] {d8d550d0-09f4-46e2-b1be-24b9634a4a4c} executeQuery: Code: 42. DB::Exception: Number of arguments for function parseDateTime32BestEffortOrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT parseDateTime32BestEffortOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTime32BestEffortOrZero([], [], []);) 2025.09.25 21:11:31.867507 [ 2707 ] {d8d550d0-09f4-46e2-b1be-24b9634a4a4c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function parseDateTime32BestEffortOrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT parseDateTime32BestEffortOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:31.980569 [ 2707 ] {0e3a1318-9331-41fa-9ca0-a1df56f07c96} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTime64BestEffort'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTime64BestEffort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTime64BestEffort([], [], []);) 2025.09.25 21:11:31.986347 [ 2707 ] {0e3a1318-9331-41fa-9ca0-a1df56f07c96} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTime64BestEffort'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTime64BestEffort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:32.108069 [ 2707 ] {03c936a1-2cdd-42d9-a785-324555dab07f} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTime64BestEffortOrNull'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTime64BestEffortOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTime64BestEffortOrNull([], [], []);) 2025.09.25 21:11:32.115845 [ 2707 ] {03c936a1-2cdd-42d9-a785-324555dab07f} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTime64BestEffortOrNull'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTime64BestEffortOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:32.291913 [ 2707 ] {bc649d79-77c2-4081-979e-a486c2286fb7} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTime64BestEffortOrZero'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTime64BestEffortOrZero([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTime64BestEffortOrZero([], [], []);) 2025.09.25 21:11:32.305899 [ 2707 ] {bc649d79-77c2-4081-979e-a486c2286fb7} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTime64BestEffortOrZero'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTime64BestEffortOrZero([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:32.333414 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:32.375262 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:32.444626 [ 2707 ] {63d7c01b-783b-4612-bb3f-28fba3f70d3d} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTime64BestEffortUS'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTime64BestEffortUS([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTime64BestEffortUS([], [], []);) 2025.09.25 21:11:32.455135 [ 2707 ] {63d7c01b-783b-4612-bb3f-28fba3f70d3d} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTime64BestEffortUS'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTime64BestEffortUS([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:32.566206 [ 2707 ] {1092f5a7-2215-4aaf-8f1d-608935cae5f5} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTime64BestEffortUSOrNull'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTime64BestEffortUSOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTime64BestEffortUSOrNull([], [], []);) 2025.09.25 21:11:32.585918 [ 2707 ] {1092f5a7-2215-4aaf-8f1d-608935cae5f5} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTime64BestEffortUSOrNull'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTime64BestEffortUSOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:32.664699 [ 2707 ] {9ea96357-98e8-49a9-8327-c30c36b2a66c} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTime64BestEffortUSOrZero'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTime64BestEffortUSOrZero([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTime64BestEffortUSOrZero([], [], []);) 2025.09.25 21:11:32.671156 [ 2707 ] {9ea96357-98e8-49a9-8327-c30c36b2a66c} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTime64BestEffortUSOrZero'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTime64BestEffortUSOrZero([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:32.771705 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:32.781346 [ 2707 ] {4c6b9e8a-0902-40da-9206-6a2782e57661} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTimeBestEffort'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTimeBestEffort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTimeBestEffort([], [], []);) 2025.09.25 21:11:32.791012 [ 2707 ] {4c6b9e8a-0902-40da-9206-6a2782e57661} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTimeBestEffort'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTimeBestEffort([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:32.813682 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:32.891391 [ 2707 ] {31886b01-0d49-4191-8d2a-8b0ef453fbc9} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTimeBestEffortOrNull'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTimeBestEffortOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTimeBestEffortOrNull([], [], []);) 2025.09.25 21:11:32.900292 [ 2707 ] {31886b01-0d49-4191-8d2a-8b0ef453fbc9} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTimeBestEffortOrNull'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTimeBestEffortOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:33.059635 [ 2707 ] {e7e38236-11bd-44f7-a5ea-ec430f9a0aef} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTimeBestEffortOrZero'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTimeBestEffortOrZero([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTimeBestEffortOrZero([], [], []);) 2025.09.25 21:11:33.065259 [ 2707 ] {e7e38236-11bd-44f7-a5ea-ec430f9a0aef} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'parseDateTimeBestEffortOrZero'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT parseDateTimeBestEffortOrZero([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:33.334341 [ 2707 ] {d82641ec-02f7-41a4-9925-9d1486fc8c02} executeQuery: Code: 42. DB::Exception: Number of arguments for function parseDateTimeBestEffortUS doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT parseDateTimeBestEffortUS([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTimeBestEffortUS([], [], []);) 2025.09.25 21:11:33.342354 [ 2707 ] {d82641ec-02f7-41a4-9925-9d1486fc8c02} TCPHandler: Code: 42. DB::Exception: Number of arguments for function parseDateTimeBestEffortUS doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT parseDateTimeBestEffortUS([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:33.451880 [ 2707 ] {edea1e5b-a889-4075-baf4-3f4102e7bb92} executeQuery: Code: 42. DB::Exception: Number of arguments for function parseDateTimeBestEffortUSOrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT parseDateTimeBestEffortUSOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTimeBestEffortUSOrNull([], [], []);) 2025.09.25 21:11:33.455653 [ 2707 ] {edea1e5b-a889-4075-baf4-3f4102e7bb92} TCPHandler: Code: 42. DB::Exception: Number of arguments for function parseDateTimeBestEffortUSOrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT parseDateTimeBestEffortUSOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:33.512484 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:33.532883 [ 2707 ] {dc0d1837-c83e-47a8-a46e-0dcfde06a7d0} executeQuery: Code: 42. DB::Exception: Number of arguments for function parseDateTimeBestEffortUSOrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT parseDateTimeBestEffortUSOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTimeBestEffortUSOrZero([], [], []);) 2025.09.25 21:11:33.555049 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:33.568841 [ 2707 ] {dc0d1837-c83e-47a8-a46e-0dcfde06a7d0} TCPHandler: Code: 42. DB::Exception: Number of arguments for function parseDateTimeBestEffortUSOrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT parseDateTimeBestEffortUSOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:33.671836 [ 2707 ] {f32630f8-ab3b-4af0-be81-375c8c64aea5} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'time' to function 'parseDateTimeInJodaSyntax'. Expected: String, got: Array(Nothing): In scope SELECT parseDateTimeInJodaSyntax([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTimeInJodaSyntax([], [], []);) 2025.09.25 21:11:33.684818 [ 2707 ] {f32630f8-ab3b-4af0-be81-375c8c64aea5} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'time' to function 'parseDateTimeInJodaSyntax'. Expected: String, got: Array(Nothing): In scope SELECT parseDateTimeInJodaSyntax([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:33.818628 [ 2707 ] {53e91c2f-0d0d-4d7e-acd9-cc0d55ca6b03} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'time' to function 'parseDateTimeInJodaSyntaxOrNull'. Expected: String, got: Array(Nothing): In scope SELECT parseDateTimeInJodaSyntaxOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTimeInJodaSyntaxOrNull([], [], []);) 2025.09.25 21:11:33.827004 [ 2707 ] {53e91c2f-0d0d-4d7e-acd9-cc0d55ca6b03} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'time' to function 'parseDateTimeInJodaSyntaxOrNull'. Expected: String, got: Array(Nothing): In scope SELECT parseDateTimeInJodaSyntaxOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:33.911791 [ 2707 ] {018a3b77-9bbb-4d07-90f3-4d4865390bf1} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'time' to function 'parseDateTimeInJodaSyntaxOrZero'. Expected: String, got: Array(Nothing): In scope SELECT parseDateTimeInJodaSyntaxOrZero([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTimeInJodaSyntaxOrZero([], [], []);) 2025.09.25 21:11:33.922642 [ 2707 ] {018a3b77-9bbb-4d07-90f3-4d4865390bf1} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'time' to function 'parseDateTimeInJodaSyntaxOrZero'. Expected: String, got: Array(Nothing): In scope SELECT parseDateTimeInJodaSyntaxOrZero([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:34.023032 [ 2707 ] {0c2402f4-9f02-48f0-9986-9d7064b47ac7} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'time' to function 'parseDateTimeOrNull'. Expected: String, got: Array(Nothing): In scope SELECT parseDateTimeOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTimeOrNull([], [], []);) 2025.09.25 21:11:34.026189 [ 2707 ] {0c2402f4-9f02-48f0-9986-9d7064b47ac7} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'time' to function 'parseDateTimeOrNull'. Expected: String, got: Array(Nothing): In scope SELECT parseDateTimeOrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:34.080608 [ 2707 ] {d907b09a-f303-4947-8cbc-94b28dec4e89} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'time' to function 'parseDateTimeOrZero'. Expected: String, got: Array(Nothing): In scope SELECT parseDateTimeOrZero([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseDateTimeOrZero([], [], []);) 2025.09.25 21:11:34.095234 [ 2707 ] {d907b09a-f303-4947-8cbc-94b28dec4e89} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'time' to function 'parseDateTimeOrZero'. Expected: String, got: Array(Nothing): In scope SELECT parseDateTimeOrZero([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:34.177829 [ 5079 ] {} test_5ynkecjo.d (ef397668-4864-4948-87da-1b70c589f025): dropAllData: got exception removing parts from disk, removing successfully removed parts from memory. 2025.09.25 21:11:34.254243 [ 2707 ] {c3969387-40e8-4fa5-94ba-58aa467d12b8} executeQuery: Code: 42. DB::Exception: Number of arguments for function parseReadableSize doesn't match: passed 3, should be 1: In scope SELECT parseReadableSize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseReadableSize([], [], []);) 2025.09.25 21:11:34.259843 [ 2707 ] {c3969387-40e8-4fa5-94ba-58aa467d12b8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function parseReadableSize doesn't match: passed 3, should be 1: In scope SELECT parseReadableSize([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:34.309565 [ 5079 ] {} DatabaseCatalog: Cannot drop table test_5ynkecjo.d (ef397668-4864-4948-87da-1b70c589f025). Will retry later.: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c47c445 8. ./contrib/llvm-project/libcxx/include/vector:547: DB::MergeTreeData::clearPartsFromFilesystem(std::vector, std::allocator>> const&, bool, std::unordered_set, std::equal_to, std::allocator>*) @ 0x000000002c47a78e 9. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::dropAllData() @ 0x000000002c4862d9 10. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::dropTableFinally(DB::DatabaseCatalog::TableMarkedAsDropped const&) @ 0x00000000286c7c77 11. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: void std::__function::__policy_invoker::__call_impl, std::allocator>>)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x00000000286d9744 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x00000000263a3ea5 13. ./contrib/llvm-project/libcxx/include/future:0: std::packaged_task::operator()() @ 0x000000001bba33ac 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:34.359731 [ 2707 ] {060f18b4-dce8-4ced-8ae8-7eaaace520d8} executeQuery: Code: 42. DB::Exception: Number of arguments for function parseReadableSizeOrNull doesn't match: passed 3, should be 1: In scope SELECT parseReadableSizeOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseReadableSizeOrNull([], [], []);) 2025.09.25 21:11:34.364637 [ 2707 ] {060f18b4-dce8-4ced-8ae8-7eaaace520d8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function parseReadableSizeOrNull doesn't match: passed 3, should be 1: In scope SELECT parseReadableSizeOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:34.432013 [ 2707 ] {8d9cf4ba-443a-4ecf-a4a1-25dc4bf431a8} executeQuery: Code: 42. DB::Exception: Number of arguments for function parseReadableSizeOrZero doesn't match: passed 3, should be 1: In scope SELECT parseReadableSizeOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseReadableSizeOrZero([], [], []);) 2025.09.25 21:11:34.437213 [ 2707 ] {8d9cf4ba-443a-4ecf-a4a1-25dc4bf431a8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function parseReadableSizeOrZero doesn't match: passed 3, should be 1: In scope SELECT parseReadableSizeOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:34.492627 [ 2707 ] {4f28b811-8098-4844-9044-3cc259102be1} executeQuery: Code: 34. DB::Exception: Number of arguments for function parseTimeDelta doesn't match: passed 3, should be 1.: In scope SELECT parseTimeDelta([], [], []). (TOO_MANY_ARGUMENTS_FOR_FUNCTION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT parseTimeDelta([], [], []);) 2025.09.25 21:11:34.495756 [ 2707 ] {4f28b811-8098-4844-9044-3cc259102be1} TCPHandler: Code: 34. DB::Exception: Number of arguments for function parseTimeDelta doesn't match: passed 3, should be 1.: In scope SELECT parseTimeDelta([], [], []). (TOO_MANY_ARGUMENTS_FOR_FUNCTION) 2025.09.25 21:11:34.675876 [ 2707 ] {228b9925-cba4-4ec5-9dc1-3eed76ae5916} executeQuery: Code: 42. DB::Exception: Number of arguments for function path doesn't match: passed 3, should be 1: In scope SELECT path([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT path([], [], []);) 2025.09.25 21:11:34.681800 [ 2707 ] {228b9925-cba4-4ec5-9dc1-3eed76ae5916} TCPHandler: Code: 42. DB::Exception: Number of arguments for function path doesn't match: passed 3, should be 1: In scope SELECT path([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:34.711433 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:34.735020 [ 2707 ] {ec3f0da3-2424-450e-9433-8d717cb30e4b} executeQuery: Code: 42. DB::Exception: Number of arguments for function pathFull doesn't match: passed 3, should be 1: In scope SELECT pathFull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT pathFull([], [], []);) 2025.09.25 21:11:34.737174 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:34.741258 [ 2707 ] {ec3f0da3-2424-450e-9433-8d717cb30e4b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function pathFull doesn't match: passed 3, should be 1: In scope SELECT pathFull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:34.818403 [ 2707 ] {ce8804b9-6576-496b-827d-345d0cfb7475} executeQuery: Code: 36. DB::Exception: The function 'percentRank' can only be used as a window function, not as an aggregate function: While executing AggregatingTransform. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT percentRank([], [], []);) 2025.09.25 21:11:34.823032 [ 2707 ] {ce8804b9-6576-496b-827d-345d0cfb7475} TCPHandler: Code: 36. DB::Exception: The function 'percentRank' can only be used as a window function, not as an aggregate function: While executing AggregatingTransform. (BAD_ARGUMENTS) 2025.09.25 21:11:35.032168 [ 2707 ] {67619dcc-f099-4b2e-bc7d-d9d20e479bcd} executeQuery: Code: 36. DB::Exception: The function 'percentRank' can only be used as a window function, not as an aggregate function: While executing AggregatingTransform. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT percent_rank([], [], []);) 2025.09.25 21:11:35.036362 [ 2707 ] {67619dcc-f099-4b2e-bc7d-d9d20e479bcd} TCPHandler: Code: 36. DB::Exception: The function 'percentRank' can only be used as a window function, not as an aggregate function: While executing AggregatingTransform. (BAD_ARGUMENTS) 2025.09.25 21:11:35.093997 [ 2707 ] {cdf31d2f-3b6b-4322-9777-05d44eb8d8ac} executeQuery: Code: 42. DB::Exception: Number of arguments for function pi doesn't match: passed 3, should be 0: In scope SELECT pi([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT pi([], [], []);) 2025.09.25 21:11:35.099633 [ 2707 ] {cdf31d2f-3b6b-4322-9777-05d44eb8d8ac} TCPHandler: Code: 42. DB::Exception: Number of arguments for function pi doesn't match: passed 3, should be 0: In scope SELECT pi([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:35.178475 [ 2707 ] {1777002a-3cd4-47b6-8270-6b3a41286ec8} executeQuery: Code: 42. DB::Exception: Number of arguments for function plus doesn't match: passed 3, should be 2: In scope SELECT plus([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT plus([], [], []);) 2025.09.25 21:11:35.187082 [ 2707 ] {1777002a-3cd4-47b6-8270-6b3a41286ec8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function plus doesn't match: passed 3, should be 2: In scope SELECT plus([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:35.238625 [ 2707 ] {1ca6621a-c1ec-4622-9da5-bf6c775c9953} executeQuery: Code: 42. DB::Exception: Number of arguments for function positiveModulo doesn't match: passed 3, should be 2: In scope SELECT pmod([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT pmod([], [], []);) 2025.09.25 21:11:35.244974 [ 2707 ] {1ca6621a-c1ec-4622-9da5-bf6c775c9953} TCPHandler: Code: 42. DB::Exception: Number of arguments for function positiveModulo doesn't match: passed 3, should be 2: In scope SELECT pmod([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:35.293537 [ 2707 ] {472cce3f-9760-49d2-a148-03ccb6244755} executeQuery: Code: 42. DB::Exception: Incorrect number of arguments of function pointInEllipses. Must be 2 for your point plus 4 * N for ellipses (x_i, y_i, a_i, b_i).: In scope SELECT pointInEllipses([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT pointInEllipses([], [], []);) 2025.09.25 21:11:35.298204 [ 2707 ] {472cce3f-9760-49d2-a148-03ccb6244755} TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments of function pointInEllipses. Must be 2 for your point plus 4 * N for ellipses (x_i, y_i, a_i, b_i).: In scope SELECT pointInEllipses([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:35.387454 [ 2707 ] {eef5e97f-8010-4abc-8e3b-595e9d3f80d9} executeQuery: Code: 43. DB::Exception: Argument 1 for function pointInPolygon must contain a tuple: In scope SELECT pointInPolygon([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT pointInPolygon([], [], []);) 2025.09.25 21:11:35.396284 [ 2707 ] {eef5e97f-8010-4abc-8e3b-595e9d3f80d9} TCPHandler: Code: 43. DB::Exception: Argument 1 for function pointInPolygon must contain a tuple: In scope SELECT pointInPolygon([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:35.424952 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:35.448622 [ 2707 ] {92c00422-95ee-4f8e-b09b-01899688332d} executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonAreaCartesian doesn't match: passed 3, should be 1: In scope SELECT polygonAreaCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT polygonAreaCartesian([], [], []);) 2025.09.25 21:11:35.454842 [ 2707 ] {92c00422-95ee-4f8e-b09b-01899688332d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonAreaCartesian doesn't match: passed 3, should be 1: In scope SELECT polygonAreaCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:35.468089 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:35.558129 [ 2707 ] {64d9da08-169a-4ddf-812a-9e4cb34e50ac} executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonAreaSpherical doesn't match: passed 3, should be 1: In scope SELECT polygonAreaSpherical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT polygonAreaSpherical([], [], []);) 2025.09.25 21:11:35.563589 [ 2707 ] {64d9da08-169a-4ddf-812a-9e4cb34e50ac} TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonAreaSpherical doesn't match: passed 3, should be 1: In scope SELECT polygonAreaSpherical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:35.610182 [ 2707 ] {a70dfb07-2cfc-4d9a-b94d-da289af1fe71} executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonConvexHullCartesian doesn't match: passed 3, should be 1: In scope SELECT polygonConvexHullCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT polygonConvexHullCartesian([], [], []);) 2025.09.25 21:11:35.614572 [ 2707 ] {a70dfb07-2cfc-4d9a-b94d-da289af1fe71} TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonConvexHullCartesian doesn't match: passed 3, should be 1: In scope SELECT polygonConvexHullCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:35.728342 [ 2707 ] {77f2c9c7-4395-42c3-b551-0cba04fa00b0} executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonPerimeterCartesian doesn't match: passed 3, should be 1: In scope SELECT polygonPerimeterCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT polygonPerimeterCartesian([], [], []);) 2025.09.25 21:11:35.733537 [ 2707 ] {77f2c9c7-4395-42c3-b551-0cba04fa00b0} TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonPerimeterCartesian doesn't match: passed 3, should be 1: In scope SELECT polygonPerimeterCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:35.803864 [ 2707 ] {32eeaf41-3093-4117-a0a5-df35076286b3} executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonPerimeterSpherical doesn't match: passed 3, should be 1: In scope SELECT polygonPerimeterSpherical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT polygonPerimeterSpherical([], [], []);) 2025.09.25 21:11:35.809549 [ 2707 ] {32eeaf41-3093-4117-a0a5-df35076286b3} TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonPerimeterSpherical doesn't match: passed 3, should be 1: In scope SELECT polygonPerimeterSpherical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:35.871557 [ 2707 ] {52560886-d667-480f-8557-e250ccb6c35c} executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsDistanceCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsDistanceCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT polygonsDistanceCartesian([], [], []);) 2025.09.25 21:11:35.876229 [ 2707 ] {52560886-d667-480f-8557-e250ccb6c35c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsDistanceCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsDistanceCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:35.928230 [ 2707 ] {d87252a8-43a4-41cf-8dec-ef54f0158f28} executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsDistanceSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsDistanceSpherical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT polygonsDistanceSpherical([], [], []);) 2025.09.25 21:11:35.998663 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:36.025525 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:36.033899 [ 2707 ] {d87252a8-43a4-41cf-8dec-ef54f0158f28} TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsDistanceSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsDistanceSpherical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:36.082304 [ 2707 ] {3bb03bb7-b266-4911-8b63-f3917ac0d03a} executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsEqualsCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsEqualsCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT polygonsEqualsCartesian([], [], []);) 2025.09.25 21:11:36.087233 [ 2707 ] {3bb03bb7-b266-4911-8b63-f3917ac0d03a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsEqualsCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsEqualsCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:36.137066 [ 2707 ] {ddce2a8d-4902-4bdd-b8a5-313f628394b1} executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsIntersectionCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsIntersectionCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT polygonsIntersectionCartesian([], [], []);) 2025.09.25 21:11:36.141917 [ 2707 ] {ddce2a8d-4902-4bdd-b8a5-313f628394b1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsIntersectionCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsIntersectionCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:36.196299 [ 2707 ] {fb86084d-654a-4cf8-8b41-cfbd48beac18} executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsIntersectionSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsIntersectionSpherical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT polygonsIntersectionSpherical([], [], []);) 2025.09.25 21:11:36.202133 [ 2707 ] {fb86084d-654a-4cf8-8b41-cfbd48beac18} TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsIntersectionSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsIntersectionSpherical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:36.250185 [ 2707 ] {1e71c011-6304-4c10-a03c-12a8c75b82a9} executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsSymDifferenceCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsSymDifferenceCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT polygonsSymDifferenceCartesian([], [], []);) 2025.09.25 21:11:36.254968 [ 2707 ] {1e71c011-6304-4c10-a03c-12a8c75b82a9} TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsSymDifferenceCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsSymDifferenceCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:36.316976 [ 2707 ] {4db14ea8-486d-4d13-a209-8f5437afe61f} executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsSymDifferenceSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsSymDifferenceSpherical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT polygonsSymDifferenceSpherical([], [], []);) 2025.09.25 21:11:36.322402 [ 2707 ] {4db14ea8-486d-4d13-a209-8f5437afe61f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsSymDifferenceSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsSymDifferenceSpherical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:36.376706 [ 2707 ] {505def33-287d-4d52-8153-66483a320158} executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsUnionCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsUnionCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT polygonsUnionCartesian([], [], []);) 2025.09.25 21:11:36.383069 [ 2707 ] {505def33-287d-4d52-8153-66483a320158} TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsUnionCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsUnionCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:36.442476 [ 2707 ] {44ef4c8e-73bf-43c3-b6de-d96a40598c0c} executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsUnionSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsUnionSpherical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT polygonsUnionSpherical([], [], []);) 2025.09.25 21:11:36.447367 [ 2707 ] {44ef4c8e-73bf-43c3-b6de-d96a40598c0c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsUnionSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsUnionSpherical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:36.504012 [ 2707 ] {83057eed-8625-4f2a-986b-ff5ba1ce04a1} executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsWithinCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsWithinCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT polygonsWithinCartesian([], [], []);) 2025.09.25 21:11:36.509145 [ 2707 ] {83057eed-8625-4f2a-986b-ff5ba1ce04a1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsWithinCartesian doesn't match: passed 3, should be 2: In scope SELECT polygonsWithinCartesian([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:36.544147 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:36.567699 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:36.569981 [ 2707 ] {b393c11c-9f98-4f8c-b7fa-d5737070ea28} executeQuery: Code: 42. DB::Exception: Number of arguments for function polygonsWithinSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsWithinSpherical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT polygonsWithinSpherical([], [], []);) 2025.09.25 21:11:36.576185 [ 2707 ] {b393c11c-9f98-4f8c-b7fa-d5737070ea28} TCPHandler: Code: 42. DB::Exception: Number of arguments for function polygonsWithinSpherical doesn't match: passed 3, should be 2: In scope SELECT polygonsWithinSpherical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:36.629444 [ 2707 ] {7b5ac2a4-aa72-4e87-ad36-948736b35302} executeQuery: Code: 42. DB::Exception: Number of arguments for function port doesn't match: passed 3, should be 1 or 2: In scope SELECT port([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT port([], [], []);) 2025.09.25 21:11:36.636958 [ 2707 ] {7b5ac2a4-aa72-4e87-ad36-948736b35302} TCPHandler: Code: 42. DB::Exception: Number of arguments for function port doesn't match: passed 3, should be 1 or 2: In scope SELECT port([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:36.693734 [ 2707 ] {c2b20911-c323-4a3a-a123-d6ea2c9f8f47} executeQuery: Code: 42. DB::Exception: Number of arguments for function portRFC doesn't match: passed 3, should be 1 or 2: In scope SELECT portRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT portRFC([], [], []);) 2025.09.25 21:11:36.700849 [ 2707 ] {c2b20911-c323-4a3a-a123-d6ea2c9f8f47} TCPHandler: Code: 42. DB::Exception: Number of arguments for function portRFC doesn't match: passed 3, should be 1 or 2: In scope SELECT portRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:36.755883 [ 2707 ] {0b0bf497-9a29-4906-9767-ef9f13a8565b} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function position: In scope SELECT position([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT position([], [], []);) 2025.09.25 21:11:36.761140 [ 2707 ] {0b0bf497-9a29-4906-9767-ef9f13a8565b} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function position: In scope SELECT position([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:36.816250 [ 2707 ] {43982ac4-1a4f-4ede-aa27-fc05939be309} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function positionCaseInsensitive: In scope SELECT positionCaseInsensitive([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT positionCaseInsensitive([], [], []);) 2025.09.25 21:11:36.830129 [ 2707 ] {43982ac4-1a4f-4ede-aa27-fc05939be309} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function positionCaseInsensitive: In scope SELECT positionCaseInsensitive([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:36.973268 [ 2707 ] {df633688-3377-4b32-8a2d-f2a6ac99578f} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function positionCaseInsensitiveUTF8: In scope SELECT positionCaseInsensitiveUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT positionCaseInsensitiveUTF8([], [], []);) 2025.09.25 21:11:36.980230 [ 2707 ] {df633688-3377-4b32-8a2d-f2a6ac99578f} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function positionCaseInsensitiveUTF8: In scope SELECT positionCaseInsensitiveUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:37.043503 [ 2707 ] {c084dd89-f6f4-49a8-b544-691d1c9ce105} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function positionUTF8: In scope SELECT positionUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT positionUTF8([], [], []);) 2025.09.25 21:11:37.048699 [ 2707 ] {c084dd89-f6f4-49a8-b544-691d1c9ce105} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function positionUTF8: In scope SELECT positionUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:37.097061 [ 2707 ] {1cfd1dbf-f342-4f9c-bddf-5c1c364367ae} executeQuery: Code: 42. DB::Exception: Number of arguments for function positiveModulo doesn't match: passed 3, should be 2: In scope SELECT positiveModulo([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT positiveModulo([], [], []);) 2025.09.25 21:11:37.115351 [ 2707 ] {1cfd1dbf-f342-4f9c-bddf-5c1c364367ae} TCPHandler: Code: 42. DB::Exception: Number of arguments for function positiveModulo doesn't match: passed 3, should be 2: In scope SELECT positiveModulo([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:37.164288 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:37.175633 [ 2707 ] {f09c2d2c-dba7-4cbb-82d2-1836065b37a7} executeQuery: Code: 42. DB::Exception: Number of arguments for function positiveModulo doesn't match: passed 3, should be 2: In scope SELECT positive_modulo([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT positive_modulo([], [], []);) 2025.09.25 21:11:37.179665 [ 2707 ] {f09c2d2c-dba7-4cbb-82d2-1836065b37a7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function positiveModulo doesn't match: passed 3, should be 2: In scope SELECT positive_modulo([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:37.185547 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:37.238443 [ 2707 ] {440d1bee-143a-4766-acb8-519da14cf4af} executeQuery: Code: 42. DB::Exception: Number of arguments for function pow doesn't match: passed 3, should be 2: In scope SELECT pow([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT pow([], [], []);) 2025.09.25 21:11:37.245518 [ 2707 ] {440d1bee-143a-4766-acb8-519da14cf4af} TCPHandler: Code: 42. DB::Exception: Number of arguments for function pow doesn't match: passed 3, should be 2: In scope SELECT pow([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:37.325046 [ 2707 ] {55b24729-1b8e-4e74-8a42-ac5d31a317be} executeQuery: Code: 42. DB::Exception: Number of arguments for function pow doesn't match: passed 3, should be 2: In scope SELECT power([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT power([], [], []);) 2025.09.25 21:11:37.330586 [ 2707 ] {55b24729-1b8e-4e74-8a42-ac5d31a317be} TCPHandler: Code: 42. DB::Exception: Number of arguments for function pow doesn't match: passed 3, should be 2: In scope SELECT power([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:37.406588 [ 2707 ] {a152d8e5-420e-433c-b151-0cece5451736} executeQuery: Code: 43. DB::Exception: The first argument type of function printf is Array(Nothing), but String type is expected: In scope SELECT printf([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT printf([], [], []);) 2025.09.25 21:11:37.411478 [ 2707 ] {a152d8e5-420e-433c-b151-0cece5451736} TCPHandler: Code: 43. DB::Exception: The first argument type of function printf is Array(Nothing), but String type is expected: In scope SELECT printf([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:37.456078 [ 2707 ] {a3e3e04e-8738-4e82-a915-bdee02734c77} executeQuery: Code: 42. DB::Exception: Number of arguments for function proportionsZTest doesn't match: passed 3, should be 6: In scope SELECT proportionsZTest([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT proportionsZTest([], [], []);) 2025.09.25 21:11:37.462006 [ 2707 ] {a3e3e04e-8738-4e82-a915-bdee02734c77} TCPHandler: Code: 42. DB::Exception: Number of arguments for function proportionsZTest doesn't match: passed 3, should be 6: In scope SELECT proportionsZTest([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:37.528817 [ 2707 ] {8237211a-5f39-4e9c-ac51-a99a91c18f4e} executeQuery: Code: 42. DB::Exception: Number of arguments for function protocol doesn't match: passed 3, should be 1: In scope SELECT protocol([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT protocol([], [], []);) 2025.09.25 21:11:37.542646 [ 2707 ] {8237211a-5f39-4e9c-ac51-a99a91c18f4e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function protocol doesn't match: passed 3, should be 1: In scope SELECT protocol([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:37.603947 [ 2707 ] {699b36e1-aae0-46c8-b88c-b995674d72c9} executeQuery: Code: 42. DB::Exception: Number of arguments for function punycodeDecode doesn't match: passed 3, should be 1: In scope SELECT punycodeDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT punycodeDecode([], [], []);) 2025.09.25 21:11:37.633364 [ 2707 ] {699b36e1-aae0-46c8-b88c-b995674d72c9} TCPHandler: Code: 42. DB::Exception: Number of arguments for function punycodeDecode doesn't match: passed 3, should be 1: In scope SELECT punycodeDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:37.693293 [ 2707 ] {e51295f0-4f73-4c27-a7f7-759cfd67d2b1} executeQuery: Code: 42. DB::Exception: Number of arguments for function punycodeEncode doesn't match: passed 3, should be 1: In scope SELECT punycodeEncode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT punycodeEncode([], [], []);) 2025.09.25 21:11:37.698301 [ 2707 ] {e51295f0-4f73-4c27-a7f7-759cfd67d2b1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function punycodeEncode doesn't match: passed 3, should be 1: In scope SELECT punycodeEncode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:37.760673 [ 2707 ] {f23fab2c-51d4-424b-90b2-f656bee0dc6b} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantile. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantile([], [], []);) 2025.09.25 21:11:37.766944 [ 2707 ] {f23fab2c-51d4-424b-90b2-f656bee0dc6b} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantile. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:37.835944 [ 2707 ] {9307c45a-63ba-47f7-8688-bf200ae24d4f} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileBFloat16. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantileBFloat16([], [], []);) 2025.09.25 21:11:37.845133 [ 2707 ] {9307c45a-63ba-47f7-8688-bf200ae24d4f} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileBFloat16. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:37.942041 [ 2707 ] {3051ef95-1f51-4130-8208-eee8016320c0} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileBFloat16Weighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantileBFloat16Weighted([], [], []);) 2025.09.25 21:11:37.947369 [ 2707 ] {3051ef95-1f51-4130-8208-eee8016320c0} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileBFloat16Weighted. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.005031 [ 2707 ] {95f96fe9-a0bf-452f-861a-d43747bf8fe6} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileDD. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantileDD([], [], []);) 2025.09.25 21:11:38.011605 [ 2707 ] {95f96fe9-a0bf-452f-861a-d43747bf8fe6} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileDD. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.148829 [ 2707 ] {d8e60730-2dd8-4323-920b-fcbd41d0e599} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileDeterministic. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantileDeterministic([], [], []);) 2025.09.25 21:11:38.151545 [ 2707 ] {d8e60730-2dd8-4323-920b-fcbd41d0e599} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileDeterministic. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.207413 [ 2707 ] {892c37f5-8462-4398-b7ab-58fad8bb733e} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExact. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantileExact([], [], []);) 2025.09.25 21:11:38.214482 [ 2707 ] {892c37f5-8462-4398-b7ab-58fad8bb733e} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExact. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.270787 [ 2707 ] {218c0cef-80d4-4178-8882-49aca42004ed} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExactExclusive. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantileExactExclusive([], [], []);) 2025.09.25 21:11:38.275918 [ 2707 ] {218c0cef-80d4-4178-8882-49aca42004ed} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExactExclusive. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.317846 [ 2707 ] {73ff0322-4ecb-4bd8-a0d9-6fb6477ac5ba} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExactHigh. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantileExactHigh([], [], []);) 2025.09.25 21:11:38.320453 [ 2707 ] {73ff0322-4ecb-4bd8-a0d9-6fb6477ac5ba} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExactHigh. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.395024 [ 2707 ] {33ffca57-0bc6-4379-a315-a1c5b6f7795f} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExactInclusive. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantileExactInclusive([], [], []);) 2025.09.25 21:11:38.398709 [ 2707 ] {33ffca57-0bc6-4379-a315-a1c5b6f7795f} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExactInclusive. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.513728 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:38.524779 [ 2707 ] {8fc7b312-ec93-440d-914a-d60c0d4e1c0e} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExactLow. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantileExactLow([], [], []);) 2025.09.25 21:11:38.527791 [ 2707 ] {8fc7b312-ec93-440d-914a-d60c0d4e1c0e} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExactLow. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.533486 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:38.573155 [ 2707 ] {5bda8f49-5851-48ae-8686-4f6635bc2cd2} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExactWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantileExactWeighted([], [], []);) 2025.09.25 21:11:38.575912 [ 2707 ] {5bda8f49-5851-48ae-8686-4f6635bc2cd2} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileExactWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.606347 [ 2707 ] {a9930c5e-3f07-4ce5-8fa5-1f8d7705af3f} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileGK. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantileGK([], [], []);) 2025.09.25 21:11:38.610299 [ 2707 ] {a9930c5e-3f07-4ce5-8fa5-1f8d7705af3f} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileGK. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.682634 [ 2707 ] {b073dc12-71e8-4bf1-ade6-69973ae03e3b} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileInterpolatedWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantileInterpolatedWeighted([], [], []);) 2025.09.25 21:11:38.686095 [ 2707 ] {b073dc12-71e8-4bf1-ade6-69973ae03e3b} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileInterpolatedWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.721212 [ 2707 ] {b7715f54-da8e-4670-9fdf-2d3e40c538a1} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileTDigest. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantileTDigest([], [], []);) 2025.09.25 21:11:38.725293 [ 2707 ] {b7715f54-da8e-4670-9fdf-2d3e40c538a1} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileTDigest. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.752611 [ 2707 ] {ae09988f-24b7-4de6-915a-2451d639d856} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileTDigestWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantileTDigestWeighted([], [], []);) 2025.09.25 21:11:38.755344 [ 2707 ] {ae09988f-24b7-4de6-915a-2451d639d856} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileTDigestWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.788074 [ 2707 ] {ad2563ca-5ede-46dc-84e0-bfa17da1617f} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileTiming. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantileTiming([], [], []);) 2025.09.25 21:11:38.791666 [ 2707 ] {ad2563ca-5ede-46dc-84e0-bfa17da1617f} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileTiming. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.839948 [ 2707 ] {bfa084a8-9d57-4cfc-a0f2-48fbb6f35c6f} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileTimingWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantileTimingWeighted([], [], []);) 2025.09.25 21:11:38.842745 [ 2707 ] {bfa084a8-9d57-4cfc-a0f2-48fbb6f35c6f} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantileTimingWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.881291 [ 2707 ] {c9ceaa16-53b3-40c9-a1cf-806847db8832} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantiles. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantiles([], [], []);) 2025.09.25 21:11:38.884925 [ 2707 ] {c9ceaa16-53b3-40c9-a1cf-806847db8832} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantiles. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.921442 [ 2707 ] {fe9c9d58-d00e-4c62-8dc0-4db8a47b33c8} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesBFloat16. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantilesBFloat16([], [], []);) 2025.09.25 21:11:38.925009 [ 2707 ] {fe9c9d58-d00e-4c62-8dc0-4db8a47b33c8} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesBFloat16. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:38.960954 [ 2707 ] {e6e47977-3af2-41b5-a82b-54ddf5e3796e} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesBFloat16Weighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantilesBFloat16Weighted([], [], []);) 2025.09.25 21:11:38.963739 [ 2707 ] {e6e47977-3af2-41b5-a82b-54ddf5e3796e} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesBFloat16Weighted. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:39.008422 [ 2707 ] {d3b660dc-9aa9-4ec0-9812-d99603ff574e} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesDD. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantilesDD([], [], []);) 2025.09.25 21:11:39.013128 [ 2707 ] {d3b660dc-9aa9-4ec0-9812-d99603ff574e} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesDD. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:39.067360 [ 2707 ] {59159ccf-bd12-4331-8dae-b1afb55ea9fe} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesDeterministic. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantilesDeterministic([], [], []);) 2025.09.25 21:11:39.076888 [ 2707 ] {59159ccf-bd12-4331-8dae-b1afb55ea9fe} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesDeterministic. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:39.131052 [ 2707 ] {4c6f4e5e-1138-46f6-9836-05aa2ce63538} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesExact. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantilesExact([], [], []);) 2025.09.25 21:11:39.135819 [ 2707 ] {4c6f4e5e-1138-46f6-9836-05aa2ce63538} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesExact. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:39.158861 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:39.201038 [ 2707 ] {6668e8f7-4aa1-46ec-bd77-444efcfa05a7} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesExactExclusive. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantilesExactExclusive([], [], []);) 2025.09.25 21:11:39.238255 [ 2707 ] {6668e8f7-4aa1-46ec-bd77-444efcfa05a7} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesExactExclusive. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:39.249926 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:39.292082 [ 2707 ] {87b24738-245c-4179-b196-bf2a09e1938e} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesExactHigh. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantilesExactHigh([], [], []);) 2025.09.25 21:11:39.300581 [ 2707 ] {87b24738-245c-4179-b196-bf2a09e1938e} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesExactHigh. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:39.405762 [ 2707 ] {d094e57b-0add-4d6f-b843-0fd95e7626a2} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesExactInclusive. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantilesExactInclusive([], [], []);) 2025.09.25 21:11:39.413421 [ 2707 ] {d094e57b-0add-4d6f-b843-0fd95e7626a2} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesExactInclusive. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:39.528289 [ 5066 ] {} test_5ynkecjo.d (ef397668-4864-4948-87da-1b70c589f025): dropAllData: got exception removing parts from disk, removing successfully removed parts from memory. 2025.09.25 21:11:39.532248 [ 5066 ] {} DatabaseCatalog: Cannot drop table test_5ynkecjo.d (ef397668-4864-4948-87da-1b70c589f025). Will retry later.: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c47c445 8. ./contrib/llvm-project/libcxx/include/vector:547: DB::MergeTreeData::clearPartsFromFilesystem(std::vector, std::allocator>> const&, bool, std::unordered_set, std::equal_to, std::allocator>*) @ 0x000000002c47a78e 9. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::dropAllData() @ 0x000000002c4862d9 10. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::dropTableFinally(DB::DatabaseCatalog::TableMarkedAsDropped const&) @ 0x00000000286c7c77 11. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: void std::__function::__policy_invoker::__call_impl, std::allocator>>)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x00000000286d9744 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x00000000263a3ea5 13. ./contrib/llvm-project/libcxx/include/future:0: std::packaged_task::operator()() @ 0x000000001bba33ac 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:39.566371 [ 2707 ] {8618638b-1a94-46bb-9aa4-cfea049e3e5f} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesExactLow. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantilesExactLow([], [], []);) 2025.09.25 21:11:39.572202 [ 2707 ] {8618638b-1a94-46bb-9aa4-cfea049e3e5f} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesExactLow. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:39.672248 [ 2707 ] {4cb81f5e-e948-4421-a738-6afbb3228002} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesExactWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantilesExactWeighted([], [], []);) 2025.09.25 21:11:39.680373 [ 2707 ] {4cb81f5e-e948-4421-a738-6afbb3228002} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesExactWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:39.733696 [ 2707 ] {574beb93-3558-4a1e-a389-2609e4130536} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesGK. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantilesGK([], [], []);) 2025.09.25 21:11:39.740598 [ 2707 ] {574beb93-3558-4a1e-a389-2609e4130536} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesGK. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:39.791527 [ 2707 ] {22353584-928c-4f3b-87c6-e7193f21d7da} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesInterpolatedWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantilesInterpolatedWeighted([], [], []);) 2025.09.25 21:11:39.797973 [ 2707 ] {22353584-928c-4f3b-87c6-e7193f21d7da} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesInterpolatedWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:39.834833 [ 2707 ] {8d72b350-2d2c-4b58-85b6-78d4131ed295} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesTDigest. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantilesTDigest([], [], []);) 2025.09.25 21:11:39.838144 [ 2707 ] {8d72b350-2d2c-4b58-85b6-78d4131ed295} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesTDigest. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:39.932965 [ 2707 ] {16515a8d-2a94-4d60-b1c3-6a216835a705} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesTDigestWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantilesTDigestWeighted([], [], []);) 2025.09.25 21:11:39.937277 [ 2707 ] {16515a8d-2a94-4d60-b1c3-6a216835a705} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesTDigestWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:39.978630 [ 2707 ] {74c5d80e-3b48-423c-9a2a-4133d71acdfa} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesTiming. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantilesTiming([], [], []);) 2025.09.25 21:11:39.982243 [ 2707 ] {74c5d80e-3b48-423c-9a2a-4133d71acdfa} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesTiming. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:40.029236 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:40.047056 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:40.122500 [ 2707 ] {6d8f0235-0ee1-433e-ae34-ca6b5734b2cb} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesTimingWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT quantilesTimingWeighted([], [], []);) 2025.09.25 21:11:40.124844 [ 2707 ] {6d8f0235-0ee1-433e-ae34-ca6b5734b2cb} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument for aggregate function quantilesTimingWeighted. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:40.255869 [ 2707 ] {c49ba765-c3f8-409f-a8de-2928a9305673} executeQuery: Code: 42. DB::Exception: Number of arguments for function queryID doesn't match: passed 3, should be 0: In scope SELECT queryID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT queryID([], [], []);) 2025.09.25 21:11:40.286030 [ 2707 ] {c49ba765-c3f8-409f-a8de-2928a9305673} TCPHandler: Code: 42. DB::Exception: Number of arguments for function queryID doesn't match: passed 3, should be 0: In scope SELECT queryID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:40.317998 [ 2707 ] {fd502deb-7a40-431b-8890-7daa90c5784e} executeQuery: Code: 42. DB::Exception: Number of arguments for function queryString doesn't match: passed 3, should be 1: In scope SELECT queryString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT queryString([], [], []);) 2025.09.25 21:11:40.330843 [ 2707 ] {fd502deb-7a40-431b-8890-7daa90c5784e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function queryString doesn't match: passed 3, should be 1: In scope SELECT queryString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:40.465901 [ 2707 ] {56af493c-795e-4d40-9484-8787d7dd7284} executeQuery: Code: 42. DB::Exception: Number of arguments for function queryStringAndFragment doesn't match: passed 3, should be 1: In scope SELECT queryStringAndFragment([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT queryStringAndFragment([], [], []);) 2025.09.25 21:11:40.469912 [ 2707 ] {56af493c-795e-4d40-9484-8787d7dd7284} TCPHandler: Code: 42. DB::Exception: Number of arguments for function queryStringAndFragment doesn't match: passed 3, should be 1: In scope SELECT queryStringAndFragment([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:40.516147 [ 2707 ] {ee35ca4d-a51d-42f0-ad0d-f19fb98f6455} executeQuery: Code: 42. DB::Exception: Number of arguments for function queryID doesn't match: passed 3, should be 0: In scope SELECT query_id([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT query_id([], [], []);) 2025.09.25 21:11:40.520322 [ 2707 ] {ee35ca4d-a51d-42f0-ad0d-f19fb98f6455} TCPHandler: Code: 42. DB::Exception: Number of arguments for function queryID doesn't match: passed 3, should be 0: In scope SELECT query_id([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:40.571113 [ 2707 ] {6f4b194e-a8bc-40c7-8dae-a8abf59ad256} executeQuery: Code: 42. DB::Exception: Number of arguments for function radians doesn't match: passed 3, should be 1: In scope SELECT radians([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT radians([], [], []);) 2025.09.25 21:11:40.575057 [ 2707 ] {6f4b194e-a8bc-40c7-8dae-a8abf59ad256} TCPHandler: Code: 42. DB::Exception: Number of arguments for function radians doesn't match: passed 3, should be 1: In scope SELECT radians([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:40.626454 [ 2707 ] {1285b7fb-6bd2-4dd7-b064-3c86b92d8e7c} executeQuery: Code: 42. DB::Exception: Number of arguments for function rand doesn't match: passed 3, should be 0 or 1.: In scope SELECT rand([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT rand([], [], []);) 2025.09.25 21:11:40.633133 [ 2707 ] {1285b7fb-6bd2-4dd7-b064-3c86b92d8e7c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function rand doesn't match: passed 3, should be 0 or 1.: In scope SELECT rand([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:40.677342 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:40.691122 [ 2707 ] {87616a3c-2953-48ae-9ffa-188806b6f19c} executeQuery: Code: 42. DB::Exception: Number of arguments for function rand doesn't match: passed 3, should be 0 or 1.: In scope SELECT rand32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT rand32([], [], []);) 2025.09.25 21:11:40.697622 [ 2707 ] {87616a3c-2953-48ae-9ffa-188806b6f19c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function rand doesn't match: passed 3, should be 0 or 1.: In scope SELECT rand32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:40.712617 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:40.751926 [ 2707 ] {4997c218-db30-4a65-8f33-f3e81fb245a7} executeQuery: Code: 42. DB::Exception: Number of arguments for function rand64 doesn't match: passed 3, should be 0 or 1.: In scope SELECT rand64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT rand64([], [], []);) 2025.09.25 21:11:40.756925 [ 2707 ] {4997c218-db30-4a65-8f33-f3e81fb245a7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function rand64 doesn't match: passed 3, should be 0 or 1.: In scope SELECT rand64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:40.803344 [ 2707 ] {82d0d9be-27e5-4803-a7d6-9e71fae4a745} executeQuery: Code: 42. DB::Exception: Wrong number of arguments for function randBernoulli. Should be 1 or 2: In scope SELECT randBernoulli([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randBernoulli([], [], []);) 2025.09.25 21:11:40.808859 [ 2707 ] {82d0d9be-27e5-4803-a7d6-9e71fae4a745} TCPHandler: Code: 42. DB::Exception: Wrong number of arguments for function randBernoulli. Should be 1 or 2: In scope SELECT randBernoulli([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:40.854334 [ 2707 ] {adbf8b9d-49f3-4685-8f10-e4cf1f0d41da} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function randBinomial, expected Float64 or integer: In scope SELECT randBinomial([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randBinomial([], [], []);) 2025.09.25 21:11:40.859263 [ 2707 ] {adbf8b9d-49f3-4685-8f10-e4cf1f0d41da} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function randBinomial, expected Float64 or integer: In scope SELECT randBinomial([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:40.910926 [ 2707 ] {95c26bce-8da4-4645-9bde-735faf17a78d} executeQuery: Code: 42. DB::Exception: Number of arguments for function randCanonical doesn't match: passed 3, should be 0 or 1.: In scope SELECT randCanonical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randCanonical([], [], []);) 2025.09.25 21:11:40.916079 [ 2707 ] {95c26bce-8da4-4645-9bde-735faf17a78d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function randCanonical doesn't match: passed 3, should be 0 or 1.: In scope SELECT randCanonical([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:40.960427 [ 2707 ] {b9eeea1a-95c0-40bf-be07-c45f787cc8cf} executeQuery: Code: 42. DB::Exception: Wrong number of arguments for function randChiSquared. Should be 1 or 2: In scope SELECT randChiSquared([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randChiSquared([], [], []);) 2025.09.25 21:11:40.996635 [ 2707 ] {b9eeea1a-95c0-40bf-be07-c45f787cc8cf} TCPHandler: Code: 42. DB::Exception: Wrong number of arguments for function randChiSquared. Should be 1 or 2: In scope SELECT randChiSquared([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:41.046327 [ 2707 ] {eabbfc68-2eb4-4f1a-9385-5a97e01a379d} executeQuery: Code: 42. DB::Exception: Number of arguments for function randConstant doesn't match: passed 3, should be 0 or 1.: In scope SELECT randConstant([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randConstant([], [], []);) 2025.09.25 21:11:41.050516 [ 2707 ] {eabbfc68-2eb4-4f1a-9385-5a97e01a379d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function randConstant doesn't match: passed 3, should be 0 or 1.: In scope SELECT randConstant([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:41.095641 [ 2707 ] {0b93ac71-a46e-4f5e-8a46-49f8ef47f109} executeQuery: Code: 42. DB::Exception: Wrong number of arguments for function randExponential. Should be 1 or 2: In scope SELECT randExponential([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randExponential([], [], []);) 2025.09.25 21:11:41.098321 [ 2707 ] {0b93ac71-a46e-4f5e-8a46-49f8ef47f109} TCPHandler: Code: 42. DB::Exception: Wrong number of arguments for function randExponential. Should be 1 or 2: In scope SELECT randExponential([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:41.138867 [ 2707 ] {aeffe8f9-230f-4bde-b5d2-bc5bd934cb2e} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function randFisherF, expected Float64 or integer: In scope SELECT randFisherF([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randFisherF([], [], []);) 2025.09.25 21:11:41.145827 [ 2707 ] {aeffe8f9-230f-4bde-b5d2-bc5bd934cb2e} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function randFisherF, expected Float64 or integer: In scope SELECT randFisherF([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:41.256164 [ 2707 ] {2b8b2e2f-d45b-42bd-9c55-0fc28aac4d59} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function randLogNormal, expected Float64 or integer: In scope SELECT randLogNormal([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randLogNormal([], [], []);) 2025.09.25 21:11:41.262180 [ 2707 ] {2b8b2e2f-d45b-42bd-9c55-0fc28aac4d59} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function randLogNormal, expected Float64 or integer: In scope SELECT randLogNormal([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:41.307006 [ 2707 ] {c59330e0-9588-4bde-a534-db1b6f3a81ac} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function randNegativeBinomial, expected Float64 or integer: In scope SELECT randNegativeBinomial([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randNegativeBinomial([], [], []);) 2025.09.25 21:11:41.311773 [ 2707 ] {c59330e0-9588-4bde-a534-db1b6f3a81ac} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function randNegativeBinomial, expected Float64 or integer: In scope SELECT randNegativeBinomial([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:41.364260 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:41.368067 [ 2707 ] {4980f020-bbaa-4dea-9b09-b14f18f865f4} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function randNormal, expected Float64 or integer: In scope SELECT randNormal([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randNormal([], [], []);) 2025.09.25 21:11:41.372090 [ 2707 ] {4980f020-bbaa-4dea-9b09-b14f18f865f4} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function randNormal, expected Float64 or integer: In scope SELECT randNormal([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:41.380701 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:41.424266 [ 2707 ] {6e16eb89-3c33-468d-ad10-ac314083567d} executeQuery: Code: 42. DB::Exception: Wrong number of arguments for function randPoisson. Should be 1 or 2: In scope SELECT randPoisson([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randPoisson([], [], []);) 2025.09.25 21:11:41.431366 [ 2707 ] {6e16eb89-3c33-468d-ad10-ac314083567d} TCPHandler: Code: 42. DB::Exception: Wrong number of arguments for function randPoisson. Should be 1 or 2: In scope SELECT randPoisson([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:41.498715 [ 2707 ] {a7d39c38-4188-498d-8702-4274957d60d5} executeQuery: Code: 42. DB::Exception: Wrong number of arguments for function randStudentT. Should be 1 or 2: In scope SELECT randStudentT([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randStudentT([], [], []);) 2025.09.25 21:11:41.504477 [ 2707 ] {a7d39c38-4188-498d-8702-4274957d60d5} TCPHandler: Code: 42. DB::Exception: Wrong number of arguments for function randStudentT. Should be 1 or 2: In scope SELECT randStudentT([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:41.550971 [ 2707 ] {68923b1b-4796-49b3-be7e-cb927b9b4573} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function randUniform, expected Float64 or integer: In scope SELECT randUniform([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randUniform([], [], []);) 2025.09.25 21:11:41.651126 [ 2707 ] {68923b1b-4796-49b3-be7e-cb927b9b4573} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function randUniform, expected Float64 or integer: In scope SELECT randUniform([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:41.696362 [ 2707 ] {f4329801-f4a9-4aab-b629-4c2b6f869335} executeQuery: Code: 42. DB::Exception: Number of arguments for function randomFixedString doesn't match: passed 3, should be 1: In scope SELECT randomFixedString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randomFixedString([], [], []);) 2025.09.25 21:11:41.701211 [ 2707 ] {f4329801-f4a9-4aab-b629-4c2b6f869335} TCPHandler: Code: 42. DB::Exception: Number of arguments for function randomFixedString doesn't match: passed 3, should be 1: In scope SELECT randomFixedString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:41.740859 [ 2707 ] {e7fe509a-4393-4a16-b87e-6807a929f1c6} executeQuery: Code: 42. DB::Exception: Function randomPrintableASCII requires at most two arguments: the size of resulting string and optional disambiguation tag: In scope SELECT randomPrintableASCII([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randomPrintableASCII([], [], []);) 2025.09.25 21:11:41.745584 [ 2707 ] {e7fe509a-4393-4a16-b87e-6807a929f1c6} TCPHandler: Code: 42. DB::Exception: Function randomPrintableASCII requires at most two arguments: the size of resulting string and optional disambiguation tag: In scope SELECT randomPrintableASCII([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:41.858006 [ 2707 ] {7b26b05c-5f76-49cb-8a47-550a9839a54b} executeQuery: Code: 42. DB::Exception: Function randomString requires at most two arguments: the size of resulting string and optional disambiguation tag: In scope SELECT randomString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randomString([], [], []);) 2025.09.25 21:11:41.861267 [ 2707 ] {7b26b05c-5f76-49cb-8a47-550a9839a54b} TCPHandler: Code: 42. DB::Exception: Function randomString requires at most two arguments: the size of resulting string and optional disambiguation tag: In scope SELECT randomString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:41.958519 [ 2707 ] {cdfb74c6-f41c-445b-9914-1ca7c695bbc2} executeQuery: Code: 42. DB::Exception: Number of arguments for function randomStringUTF8 doesn't match: passed 3, should be 1: In scope SELECT randomStringUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT randomStringUTF8([], [], []);) 2025.09.25 21:11:41.971626 [ 2707 ] {cdfb74c6-f41c-445b-9914-1ca7c695bbc2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function randomStringUTF8 doesn't match: passed 3, should be 1: In scope SELECT randomStringUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:42.034347 [ 2707 ] {52357668-8889-4b43-a53e-e0a6296e0a79} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function range: In scope SELECT range([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT range([], [], []);) 2025.09.25 21:11:42.040528 [ 2707 ] {52357668-8889-4b43-a53e-e0a6296e0a79} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function range: In scope SELECT range([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:42.135667 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:42.160640 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:42.163111 [ 2707 ] {c53ef597-372c-42cf-8f9d-5dae48d508f5} executeQuery: Code: 36. DB::Exception: The function 'rank' can only be used as a window function, not as an aggregate function: While executing AggregatingTransform. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT rank([], [], []);) 2025.09.25 21:11:42.168082 [ 2707 ] {c53ef597-372c-42cf-8f9d-5dae48d508f5} TCPHandler: Code: 36. DB::Exception: The function 'rank' can only be used as a window function, not as an aggregate function: While executing AggregatingTransform. (BAD_ARGUMENTS) 2025.09.25 21:11:42.224569 [ 2707 ] {bc431b27-4f6d-4fb1-a611-c57e77ebe240} executeQuery: Code: 42. DB::Exception: Aggregate function rankCorr requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT rankCorr([], [], []);) 2025.09.25 21:11:42.228156 [ 2707 ] {bc431b27-4f6d-4fb1-a611-c57e77ebe240} TCPHandler: Code: 42. DB::Exception: Aggregate function rankCorr requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:42.273619 [ 2707 ] {503b2830-3813-41fa-8780-0e0ece20a9ee} executeQuery: Code: 42. DB::Exception: Number of arguments for function readWKTLineString doesn't match: passed 3, should be 1: In scope SELECT readWKTLineString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT readWKTLineString([], [], []);) 2025.09.25 21:11:42.277580 [ 2707 ] {503b2830-3813-41fa-8780-0e0ece20a9ee} TCPHandler: Code: 42. DB::Exception: Number of arguments for function readWKTLineString doesn't match: passed 3, should be 1: In scope SELECT readWKTLineString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:42.326957 [ 2707 ] {6da0d2d5-bc44-4cd8-9250-aa6101236fa7} executeQuery: Code: 42. DB::Exception: Number of arguments for function readWKTMultiLineString doesn't match: passed 3, should be 1: In scope SELECT readWKTMultiLineString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT readWKTMultiLineString([], [], []);) 2025.09.25 21:11:42.351153 [ 2707 ] {6da0d2d5-bc44-4cd8-9250-aa6101236fa7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function readWKTMultiLineString doesn't match: passed 3, should be 1: In scope SELECT readWKTMultiLineString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:42.420952 [ 2707 ] {6ce59fe3-51db-4656-b6ba-63bdf5150e46} executeQuery: Code: 42. DB::Exception: Number of arguments for function readWKTMultiPolygon doesn't match: passed 3, should be 1: In scope SELECT readWKTMultiPolygon([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT readWKTMultiPolygon([], [], []);) 2025.09.25 21:11:42.425945 [ 2707 ] {6ce59fe3-51db-4656-b6ba-63bdf5150e46} TCPHandler: Code: 42. DB::Exception: Number of arguments for function readWKTMultiPolygon doesn't match: passed 3, should be 1: In scope SELECT readWKTMultiPolygon([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:42.466080 [ 2707 ] {40f8b86c-bb14-4254-9bb3-15e9a0e47cbe} executeQuery: Code: 42. DB::Exception: Number of arguments for function readWKTPoint doesn't match: passed 3, should be 1: In scope SELECT readWKTPoint([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT readWKTPoint([], [], []);) 2025.09.25 21:11:42.472159 [ 2707 ] {40f8b86c-bb14-4254-9bb3-15e9a0e47cbe} TCPHandler: Code: 42. DB::Exception: Number of arguments for function readWKTPoint doesn't match: passed 3, should be 1: In scope SELECT readWKTPoint([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:42.506287 [ 2707 ] {797b105b-4366-4065-a500-51f8118a09b5} executeQuery: Code: 42. DB::Exception: Number of arguments for function readWKTPolygon doesn't match: passed 3, should be 1: In scope SELECT readWKTPolygon([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT readWKTPolygon([], [], []);) 2025.09.25 21:11:42.509582 [ 2707 ] {797b105b-4366-4065-a500-51f8118a09b5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function readWKTPolygon doesn't match: passed 3, should be 1: In scope SELECT readWKTPolygon([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:42.542468 [ 2707 ] {d8c0f8fb-d0cf-454b-bead-a479967a04c7} executeQuery: Code: 42. DB::Exception: Number of arguments for function readWKTRing doesn't match: passed 3, should be 1: In scope SELECT readWKTRing([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT readWKTRing([], [], []);) 2025.09.25 21:11:42.547013 [ 2707 ] {d8c0f8fb-d0cf-454b-bead-a479967a04c7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function readWKTRing doesn't match: passed 3, should be 1: In scope SELECT readWKTRing([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:42.577974 [ 2707 ] {f0ec8cf3-915e-473e-a17a-74773ec6e2f3} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'regexpExtract'. Expected: String, got: Array(Nothing): In scope SELECT regexpExtract([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT regexpExtract([], [], []);) 2025.09.25 21:11:42.584661 [ 2707 ] {f0ec8cf3-915e-473e-a17a-74773ec6e2f3} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'regexpExtract'. Expected: String, got: Array(Nothing): In scope SELECT regexpExtract([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:42.628094 [ 2707 ] {9b74df70-81e4-4eac-979d-a05db653cb3c} executeQuery: Code: 42. DB::Exception: Number of arguments for function regexpQuoteMeta doesn't match: passed 3, should be 1: In scope SELECT regexpQuoteMeta([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT regexpQuoteMeta([], [], []);) 2025.09.25 21:11:42.631579 [ 2707 ] {9b74df70-81e4-4eac-979d-a05db653cb3c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function regexpQuoteMeta doesn't match: passed 3, should be 1: In scope SELECT regexpQuoteMeta([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:42.687431 [ 2707 ] {1ebe3b60-57fa-42d3-81ff-75b443385269} executeQuery: Code: 42. DB::Exception: Number of arguments for function regionHierarchy doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionHierarchy([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT regionHierarchy([], [], []);) 2025.09.25 21:11:42.691456 [ 2707 ] {1ebe3b60-57fa-42d3-81ff-75b443385269} TCPHandler: Code: 42. DB::Exception: Number of arguments for function regionHierarchy doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionHierarchy([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:42.781222 [ 2707 ] {1638f569-e697-4689-a985-18a4e67ffb90} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function regionIn (must be UInt32): In scope SELECT regionIn([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT regionIn([], [], []);) 2025.09.25 21:11:42.785670 [ 2707 ] {1638f569-e697-4689-a985-18a4e67ffb90} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function regionIn (must be UInt32): In scope SELECT regionIn([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:42.836185 [ 2707 ] {fdb1a7af-3722-496b-b021-e970aff48f49} executeQuery: Code: 42. DB::Exception: Number of arguments for function regionToArea doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionToArea([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT regionToArea([], [], []);) 2025.09.25 21:11:42.840722 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:42.848506 [ 2707 ] {fdb1a7af-3722-496b-b021-e970aff48f49} TCPHandler: Code: 42. DB::Exception: Number of arguments for function regionToArea doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionToArea([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:42.861369 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:42.885941 [ 2707 ] {14ba22d4-b45d-4708-99a5-478d65e01ae4} executeQuery: Code: 42. DB::Exception: Number of arguments for function regionToCity doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionToCity([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT regionToCity([], [], []);) 2025.09.25 21:11:42.890601 [ 2707 ] {14ba22d4-b45d-4708-99a5-478d65e01ae4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function regionToCity doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionToCity([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:42.942638 [ 2707 ] {0235d1f2-827c-4f5c-ab61-c989b36ffae4} executeQuery: Code: 42. DB::Exception: Number of arguments for function regionToContinent doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionToContinent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT regionToContinent([], [], []);) 2025.09.25 21:11:42.946859 [ 2707 ] {0235d1f2-827c-4f5c-ab61-c989b36ffae4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function regionToContinent doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionToContinent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:43.012772 [ 2707 ] {90575394-eb16-4591-ad39-8c7974ff91ad} executeQuery: Code: 42. DB::Exception: Number of arguments for function regionToCountry doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionToCountry([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT regionToCountry([], [], []);) 2025.09.25 21:11:43.018178 [ 2707 ] {90575394-eb16-4591-ad39-8c7974ff91ad} TCPHandler: Code: 42. DB::Exception: Number of arguments for function regionToCountry doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionToCountry([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:43.065295 [ 2707 ] {2d2b3713-92b5-4b90-a31b-fd0b005efc57} executeQuery: Code: 42. DB::Exception: Number of arguments for function regionToDistrict doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionToDistrict([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT regionToDistrict([], [], []);) 2025.09.25 21:11:43.077835 [ 2707 ] {2d2b3713-92b5-4b90-a31b-fd0b005efc57} TCPHandler: Code: 42. DB::Exception: Number of arguments for function regionToDistrict doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionToDistrict([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:43.127801 [ 2707 ] {2df3626a-9efe-473e-a4e6-b5002301152a} executeQuery: Code: 42. DB::Exception: Number of arguments for function regionToName doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionToName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT regionToName([], [], []);) 2025.09.25 21:11:43.132058 [ 2707 ] {2df3626a-9efe-473e-a4e6-b5002301152a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function regionToName doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionToName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:43.170745 [ 2707 ] {c98f44e3-e7c0-4633-9864-1b0e26d904f4} executeQuery: Code: 42. DB::Exception: Number of arguments for function regionToPopulation doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionToPopulation([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT regionToPopulation([], [], []);) 2025.09.25 21:11:43.174349 [ 2707 ] {c98f44e3-e7c0-4633-9864-1b0e26d904f4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function regionToPopulation doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionToPopulation([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:43.230006 [ 2707 ] {99310288-4313-4ab5-8aa5-fedbf58d2d42} executeQuery: Code: 42. DB::Exception: Number of arguments for function regionToTopContinent doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionToTopContinent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT regionToTopContinent([], [], []);) 2025.09.25 21:11:43.234411 [ 2707 ] {99310288-4313-4ab5-8aa5-fedbf58d2d42} TCPHandler: Code: 42. DB::Exception: Number of arguments for function regionToTopContinent doesn't match: passed 3, should be 1 or 2.: In scope SELECT regionToTopContinent([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:43.273963 [ 2707 ] {eecd3b9a-da5d-4693-8294-4ddc88274597} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpret doesn't match: passed 3, should be 2: In scope SELECT reinterpret([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpret([], [], []);) 2025.09.25 21:11:43.289624 [ 2707 ] {eecd3b9a-da5d-4693-8294-4ddc88274597} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpret doesn't match: passed 3, should be 2: In scope SELECT reinterpret([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:43.354634 [ 2707 ] {435563d3-6255-4662-b2ef-72a9292b0e98} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsDate doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsDate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsDate([], [], []);) 2025.09.25 21:11:43.362814 [ 2707 ] {435563d3-6255-4662-b2ef-72a9292b0e98} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsDate doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsDate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:43.450775 [ 2707 ] {5977f038-d49a-4f1a-aeec-8d4dd120f103} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsDateTime doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsDateTime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsDateTime([], [], []);) 2025.09.25 21:11:43.464032 [ 2707 ] {5977f038-d49a-4f1a-aeec-8d4dd120f103} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsDateTime doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsDateTime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:43.521643 [ 2707 ] {0e03f006-1dea-498e-8783-656cbda02ab1} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsFixedString doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsFixedString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsFixedString([], [], []);) 2025.09.25 21:11:43.525274 [ 2707 ] {0e03f006-1dea-498e-8783-656cbda02ab1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsFixedString doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsFixedString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:43.608833 [ 2707 ] {9f6898a5-eb9d-492f-9fe5-a90cbaf8d664} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsFloat32 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsFloat32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsFloat32([], [], []);) 2025.09.25 21:11:43.616336 [ 2707 ] {9f6898a5-eb9d-492f-9fe5-a90cbaf8d664} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsFloat32 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsFloat32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:43.658441 [ 2707 ] {c9d9c46b-d4cb-4b51-8eac-6bfd0900353e} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsFloat64 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsFloat64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsFloat64([], [], []);) 2025.09.25 21:11:43.662707 [ 2707 ] {c9d9c46b-d4cb-4b51-8eac-6bfd0900353e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsFloat64 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsFloat64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:43.721794 [ 2707 ] {8dee8125-7de6-4a01-859f-4903fa343878} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt128 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt128([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsInt128([], [], []);) 2025.09.25 21:11:43.725782 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:43.726722 [ 2707 ] {8dee8125-7de6-4a01-859f-4903fa343878} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt128 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt128([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:43.745072 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:43.778874 [ 2707 ] {16a99682-69d8-4ac9-9cd7-e7a93f4dc744} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt16 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt16([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsInt16([], [], []);) 2025.09.25 21:11:43.783794 [ 2707 ] {16a99682-69d8-4ac9-9cd7-e7a93f4dc744} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt16 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt16([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:43.821687 [ 2707 ] {3fcd0d07-1973-4a38-a51f-5dc32e3a475f} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt256 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt256([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsInt256([], [], []);) 2025.09.25 21:11:43.837148 [ 2707 ] {3fcd0d07-1973-4a38-a51f-5dc32e3a475f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt256 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt256([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:43.910894 [ 2707 ] {73e93883-bd46-4d91-a32d-41279daa0e5a} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt32 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsInt32([], [], []);) 2025.09.25 21:11:43.914276 [ 2707 ] {73e93883-bd46-4d91-a32d-41279daa0e5a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt32 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:43.957173 [ 2707 ] {53502c29-5b64-4326-b083-1416be510607} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt64 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsInt64([], [], []);) 2025.09.25 21:11:43.960186 [ 2707 ] {53502c29-5b64-4326-b083-1416be510607} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt64 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:44.012351 [ 2707 ] {c8686c54-4c71-4ddd-a9ab-db68211a7470} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt8 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsInt8([], [], []);) 2025.09.25 21:11:44.018909 [ 2707 ] {c8686c54-4c71-4ddd-a9ab-db68211a7470} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsInt8 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsInt8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:44.132101 [ 2707 ] {5d24a047-7946-4e6d-890d-e08f0f0ef437} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsString doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsString([], [], []);) 2025.09.25 21:11:44.136272 [ 2707 ] {5d24a047-7946-4e6d-890d-e08f0f0ef437} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsString doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:44.186895 [ 2707 ] {1319da37-b686-41bf-aadf-8487ea1602b0} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt128 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt128([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsUInt128([], [], []);) 2025.09.25 21:11:44.194211 [ 2707 ] {1319da37-b686-41bf-aadf-8487ea1602b0} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt128 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt128([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:44.219716 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:44.243040 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:44.263037 [ 2707 ] {914f53e7-7be5-4693-8023-d46f30cd607e} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt16 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt16([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsUInt16([], [], []);) 2025.09.25 21:11:44.269700 [ 2707 ] {914f53e7-7be5-4693-8023-d46f30cd607e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt16 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt16([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:44.392818 [ 2707 ] {917dedd8-2647-4518-9524-8033031abd2a} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt256 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt256([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsUInt256([], [], []);) 2025.09.25 21:11:44.396080 [ 2707 ] {917dedd8-2647-4518-9524-8033031abd2a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt256 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt256([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:44.440976 [ 2707 ] {d8955a31-7d9c-43d6-aaf5-6e53db937d91} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt32 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsUInt32([], [], []);) 2025.09.25 21:11:44.444663 [ 2707 ] {d8955a31-7d9c-43d6-aaf5-6e53db937d91} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt32 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:44.512044 [ 2707 ] {d2eeb369-c3eb-46c0-afd1-225502376a86} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt64 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsUInt64([], [], []);) 2025.09.25 21:11:44.526198 [ 2707 ] {d2eeb369-c3eb-46c0-afd1-225502376a86} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt64 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:44.646356 [ 2707 ] {627ca137-bb9c-4e45-9b03-018ceecb393e} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt8 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsUInt8([], [], []);) 2025.09.25 21:11:44.660247 [ 2707 ] {627ca137-bb9c-4e45-9b03-018ceecb393e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUInt8 doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUInt8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:44.694069 [ 3365 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {02260661-95d0-4eca-a73d-2543d99ce5b9::202509_103_108_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:44.708363 [ 2707 ] {edd2e761-cec6-44ad-a8ab-2edc09004af5} executeQuery: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUUID doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUUID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reinterpretAsUUID([], [], []);) 2025.09.25 21:11:44.716733 [ 2707 ] {edd2e761-cec6-44ad-a8ab-2edc09004af5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reinterpretAsUUID doesn't match: passed 3, should be 1: In scope SELECT reinterpretAsUUID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:44.880753 [ 2707 ] {4f703e1e-b1fe-4874-b7e7-a2f53931f584} executeQuery: Code: 42. DB::Exception: Number of arguments for function repeat doesn't match: passed 3, should be 2: In scope SELECT repeat([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT repeat([], [], []);) 2025.09.25 21:11:44.954242 [ 2707 ] {4f703e1e-b1fe-4874-b7e7-a2f53931f584} TCPHandler: Code: 42. DB::Exception: Number of arguments for function repeat doesn't match: passed 3, should be 2: In scope SELECT repeat([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:45.008159 [ 2707 ] {8ba196bd-2b0c-49e5-bb3b-909fea51f07a} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'replaceAll'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT replace([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT replace([], [], []);) 2025.09.25 21:11:45.021097 [ 2707 ] {8ba196bd-2b0c-49e5-bb3b-909fea51f07a} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'replaceAll'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT replace([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:45.162307 [ 2707 ] {c5e3265a-4eb4-4d7e-8f1f-df7efe6f53da} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'replaceAll'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT replaceAll([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT replaceAll([], [], []);) 2025.09.25 21:11:45.164877 [ 2707 ] {c5e3265a-4eb4-4d7e-8f1f-df7efe6f53da} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'replaceAll'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT replaceAll([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:45.265371 [ 2707 ] {ad2b00dc-db16-40f1-a4c3-f173e50c9cef} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'replaceOne'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT replaceOne([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT replaceOne([], [], []);) 2025.09.25 21:11:45.272618 [ 2707 ] {ad2b00dc-db16-40f1-a4c3-f173e50c9cef} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'replaceOne'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT replaceOne([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:45.316138 [ 2707 ] {a64acd5f-a250-41ed-a834-852363fcada8} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'replaceRegexpAll'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT replaceRegexpAll([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT replaceRegexpAll([], [], []);) 2025.09.25 21:11:45.324300 [ 2707 ] {a64acd5f-a250-41ed-a834-852363fcada8} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'replaceRegexpAll'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT replaceRegexpAll([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:45.417942 [ 2707 ] {9c41b5d7-3ea7-4407-9fdd-8d158c67bfa8} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'replaceRegexpOne'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT replaceRegexpOne([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT replaceRegexpOne([], [], []);) 2025.09.25 21:11:45.423839 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:45.433535 [ 2707 ] {9c41b5d7-3ea7-4407-9fdd-8d158c67bfa8} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'replaceRegexpOne'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT replaceRegexpOne([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:45.451559 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:45.620247 [ 2707 ] {6e9469a2-5083-4a08-b0df-42efa26c75b3} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument 0 of aggregate function retention, must be UInt8. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT retention([], [], []);) 2025.09.25 21:11:45.631981 [ 2707 ] {6e9469a2-5083-4a08-b0df-42efa26c75b3} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument 0 of aggregate function retention, must be UInt8. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:45.690691 [ 2707 ] {9b717aed-66f5-443c-ad06-52e6133d555c} executeQuery: Code: 42. DB::Exception: Number of arguments for function reverse doesn't match: passed 3, should be 1: In scope SELECT reverse([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reverse([], [], []);) 2025.09.25 21:11:45.695187 [ 2707 ] {9b717aed-66f5-443c-ad06-52e6133d555c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reverse doesn't match: passed 3, should be 1: In scope SELECT reverse([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:45.745960 [ 2707 ] {12a986c2-ca9b-47f3-bf0b-d7c0d0bba802} executeQuery: Code: 42. DB::Exception: Number of arguments for function reverseUTF8 doesn't match: passed 3, should be 1: In scope SELECT reverseUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT reverseUTF8([], [], []);) 2025.09.25 21:11:45.749969 [ 2707 ] {12a986c2-ca9b-47f3-bf0b-d7c0d0bba802} TCPHandler: Code: 42. DB::Exception: Number of arguments for function reverseUTF8 doesn't match: passed 3, should be 1: In scope SELECT reverseUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:45.794513 [ 2707 ] {f22afed8-fad2-4f97-ae70-0e5a0c0c7b19} executeQuery: Code: 42. DB::Exception: Number of arguments for function revision doesn't match: passed 3, should be 0: In scope SELECT revision([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT revision([], [], []);) 2025.09.25 21:11:45.797608 [ 2707 ] {f22afed8-fad2-4f97-ae70-0e5a0c0c7b19} TCPHandler: Code: 42. DB::Exception: Number of arguments for function revision doesn't match: passed 3, should be 0: In scope SELECT revision([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:45.862970 [ 2707 ] {710e4431-8f15-49bd-860f-6be225b5c41b} executeQuery: Code: 42. DB::Exception: Number of arguments for function right doesn't match: passed 3, should be 2: In scope SELECT right([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT right([], [], []);) 2025.09.25 21:11:45.872581 [ 2707 ] {710e4431-8f15-49bd-860f-6be225b5c41b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function right doesn't match: passed 3, should be 2: In scope SELECT right([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:46.033478 [ 2707 ] {53f742e9-8ca5-49cf-8797-9c27225b507f} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of the first argument of function rightPad, should be string: In scope SELECT rightPad([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT rightPad([], [], []);) 2025.09.25 21:11:46.039863 [ 2707 ] {53f742e9-8ca5-49cf-8797-9c27225b507f} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of the first argument of function rightPad, should be string: In scope SELECT rightPad([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:46.083570 [ 2707 ] {7f28e15d-8608-469c-94ce-d75d0ee06eeb} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of the first argument of function rightPadUTF8, should be string: In scope SELECT rightPadUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT rightPadUTF8([], [], []);) 2025.09.25 21:11:46.092824 [ 2707 ] {7f28e15d-8608-469c-94ce-d75d0ee06eeb} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of the first argument of function rightPadUTF8, should be string: In scope SELECT rightPadUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:46.136694 [ 2707 ] {ae2002b8-8367-478b-9b11-541a3c68d165} executeQuery: Code: 42. DB::Exception: Number of arguments for function rightUTF8 doesn't match: passed 3, should be 2: In scope SELECT rightUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT rightUTF8([], [], []);) 2025.09.25 21:11:46.140777 [ 2707 ] {ae2002b8-8367-478b-9b11-541a3c68d165} TCPHandler: Code: 42. DB::Exception: Number of arguments for function rightUTF8 doesn't match: passed 3, should be 2: In scope SELECT rightUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:46.168269 [ 2707 ] {b1eeace6-8f03-45a8-b7c0-97f85207c280} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'round'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT round([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT round([], [], []);) 2025.09.25 21:11:46.170711 [ 2707 ] {b1eeace6-8f03-45a8-b7c0-97f85207c280} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'round'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT round([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:46.205689 [ 2707 ] {1f7b0980-8773-4978-9028-88eeac21ec46} executeQuery: Code: 42. DB::Exception: Number of arguments for function roundAge doesn't match: passed 3, should be 1: In scope SELECT roundAge([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT roundAge([], [], []);) 2025.09.25 21:11:46.208793 [ 2707 ] {1f7b0980-8773-4978-9028-88eeac21ec46} TCPHandler: Code: 42. DB::Exception: Number of arguments for function roundAge doesn't match: passed 3, should be 1: In scope SELECT roundAge([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:46.240058 [ 2707 ] {7b7523f2-9da9-4309-8cf8-bc421d49ef62} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'roundBankers'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT roundBankers([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT roundBankers([], [], []);) 2025.09.25 21:11:46.243573 [ 2707 ] {7b7523f2-9da9-4309-8cf8-bc421d49ef62} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'roundBankers'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT roundBankers([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:46.284017 [ 2707 ] {03bbdf70-cae5-40b2-9815-2568c3cb2446} executeQuery: Code: 42. DB::Exception: Number of arguments for function roundDown doesn't match: passed 3, should be 2: In scope SELECT roundDown([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT roundDown([], [], []);) 2025.09.25 21:11:46.287851 [ 2707 ] {03bbdf70-cae5-40b2-9815-2568c3cb2446} TCPHandler: Code: 42. DB::Exception: Number of arguments for function roundDown doesn't match: passed 3, should be 2: In scope SELECT roundDown([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:46.318578 [ 2707 ] {09900a46-dd6b-46a7-8f58-4baeeb521386} executeQuery: Code: 42. DB::Exception: Number of arguments for function roundDuration doesn't match: passed 3, should be 1: In scope SELECT roundDuration([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT roundDuration([], [], []);) 2025.09.25 21:11:46.330830 [ 2707 ] {09900a46-dd6b-46a7-8f58-4baeeb521386} TCPHandler: Code: 42. DB::Exception: Number of arguments for function roundDuration doesn't match: passed 3, should be 1: In scope SELECT roundDuration([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:46.374703 [ 2707 ] {e0d30f5d-63a8-4534-9181-738079a6badc} executeQuery: Code: 42. DB::Exception: Number of arguments for function roundToExp2 doesn't match: passed 3, should be 1: In scope SELECT roundToExp2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT roundToExp2([], [], []);) 2025.09.25 21:11:46.377460 [ 2707 ] {e0d30f5d-63a8-4534-9181-738079a6badc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function roundToExp2 doesn't match: passed 3, should be 1: In scope SELECT roundToExp2([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:46.406165 [ 2707 ] {3f3b22d4-60c4-4d71-99fb-da39678ad113} executeQuery: Code: 42. DB::Exception: Number of arguments for function rowNumberInAllBlocks doesn't match: passed 3, should be 0: In scope SELECT rowNumberInAllBlocks([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT rowNumberInAllBlocks([], [], []);) 2025.09.25 21:11:46.410033 [ 2707 ] {3f3b22d4-60c4-4d71-99fb-da39678ad113} TCPHandler: Code: 42. DB::Exception: Number of arguments for function rowNumberInAllBlocks doesn't match: passed 3, should be 0: In scope SELECT rowNumberInAllBlocks([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:46.441176 [ 2707 ] {a6d58631-b442-44ee-a9c9-2fb5f6a05e95} executeQuery: Code: 42. DB::Exception: Number of arguments for function rowNumberInBlock doesn't match: passed 3, should be 0: In scope SELECT rowNumberInBlock([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT rowNumberInBlock([], [], []);) 2025.09.25 21:11:46.444043 [ 2707 ] {a6d58631-b442-44ee-a9c9-2fb5f6a05e95} TCPHandler: Code: 42. DB::Exception: Number of arguments for function rowNumberInBlock doesn't match: passed 3, should be 0: In scope SELECT rowNumberInBlock([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:46.503186 [ 2707 ] {01ee476e-c992-4bde-9fa2-240a2badbc4e} executeQuery: Code: 36. DB::Exception: The function 'row_number' can only be used as a window function, not as an aggregate function: While executing AggregatingTransform. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT row_number([], [], []);) 2025.09.25 21:11:46.506623 [ 2707 ] {01ee476e-c992-4bde-9fa2-240a2badbc4e} TCPHandler: Code: 36. DB::Exception: The function 'row_number' can only be used as a window function, not as an aggregate function: While executing AggregatingTransform. (BAD_ARGUMENTS) 2025.09.25 21:11:46.543575 [ 2707 ] {890715c4-8873-40c0-b19d-b522fa28aa64} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of the first argument of function rightPad, should be string: In scope SELECT rpad([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT rpad([], [], []);) 2025.09.25 21:11:46.548356 [ 2707 ] {890715c4-8873-40c0-b19d-b522fa28aa64} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of the first argument of function rightPad, should be string: In scope SELECT rpad([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:46.592061 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:46.611464 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:46.679808 [ 2707 ] {910b1914-f49c-48aa-846b-cb4f63eb185d} executeQuery: Code: 721. DB::Exception: Function runningAccumulate is deprecated since its usage is error-prone (see docs).Please use proper window function or set `allow_deprecated_error_prone_window_functions` setting to enable it. (DEPRECATED_FUNCTION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT runningAccumulate([], [], []);) 2025.09.25 21:11:46.683913 [ 2707 ] {910b1914-f49c-48aa-846b-cb4f63eb185d} TCPHandler: Code: 721. DB::Exception: Function runningAccumulate is deprecated since its usage is error-prone (see docs).Please use proper window function or set `allow_deprecated_error_prone_window_functions` setting to enable it. (DEPRECATED_FUNCTION) 2025.09.25 21:11:46.827281 [ 2707 ] {505813f2-fe17-460e-a38c-933231d4cf56} executeQuery: Code: 42. DB::Exception: Number of arguments for function runningConcurrency doesn't match: passed 3, should be 2: In scope SELECT runningConcurrency([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT runningConcurrency([], [], []);) 2025.09.25 21:11:46.834497 [ 2707 ] {505813f2-fe17-460e-a38c-933231d4cf56} TCPHandler: Code: 42. DB::Exception: Number of arguments for function runningConcurrency doesn't match: passed 3, should be 2: In scope SELECT runningConcurrency([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:46.918014 [ 2707 ] {79ebe35c-1c7f-4787-99a6-0447f9d6c17b} executeQuery: Code: 721. DB::Exception: Function runningDifference is deprecated since its usage is error-prone (see docs).Please use proper window function or set `allow_deprecated_error_prone_window_functions` setting to enable it. (DEPRECATED_FUNCTION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT runningDifference([], [], []);) 2025.09.25 21:11:46.924195 [ 2707 ] {79ebe35c-1c7f-4787-99a6-0447f9d6c17b} TCPHandler: Code: 721. DB::Exception: Function runningDifference is deprecated since its usage is error-prone (see docs).Please use proper window function or set `allow_deprecated_error_prone_window_functions` setting to enable it. (DEPRECATED_FUNCTION) 2025.09.25 21:11:46.970540 [ 2707 ] {52830686-69a0-4be5-8a5f-505587b736e3} executeQuery: Code: 721. DB::Exception: Function runningDifferenceStartingWithFirstValue is deprecated since its usage is error-prone (see docs).Please use proper window function or set `allow_deprecated_error_prone_window_functions` setting to enable it. (DEPRECATED_FUNCTION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT runningDifferenceStartingWithFirstValue([], [], []);) 2025.09.25 21:11:46.974241 [ 2707 ] {52830686-69a0-4be5-8a5f-505587b736e3} TCPHandler: Code: 721. DB::Exception: Function runningDifferenceStartingWithFirstValue is deprecated since its usage is error-prone (see docs).Please use proper window function or set `allow_deprecated_error_prone_window_functions` setting to enable it. (DEPRECATED_FUNCTION) 2025.09.25 21:11:47.013324 [ 2707 ] {3ebbf39b-f6cc-49bd-beec-088607868241} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument 1 of function s2CapContains. Must be UInt64: In scope SELECT s2CapContains([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT s2CapContains([], [], []);) 2025.09.25 21:11:47.016817 [ 2707 ] {3ebbf39b-f6cc-49bd-beec-088607868241} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument 1 of function s2CapContains. Must be UInt64: In scope SELECT s2CapContains([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:47.142806 [ 2707 ] {59ef2a6e-e9c2-49ef-a862-89855893a402} executeQuery: Code: 42. DB::Exception: Number of arguments for function s2CapUnion doesn't match: passed 3, should be 4: In scope SELECT s2CapUnion([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT s2CapUnion([], [], []);) 2025.09.25 21:11:47.154253 [ 2707 ] {59ef2a6e-e9c2-49ef-a862-89855893a402} TCPHandler: Code: 42. DB::Exception: Number of arguments for function s2CapUnion doesn't match: passed 3, should be 4: In scope SELECT s2CapUnion([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:47.215348 [ 2707 ] {802fe5d6-d26b-4bf6-b7ff-d27b145ac4be} executeQuery: Code: 42. DB::Exception: Number of arguments for function s2CellsIntersect doesn't match: passed 3, should be 2: In scope SELECT s2CellsIntersect([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT s2CellsIntersect([], [], []);) 2025.09.25 21:11:47.220633 [ 2707 ] {802fe5d6-d26b-4bf6-b7ff-d27b145ac4be} TCPHandler: Code: 42. DB::Exception: Number of arguments for function s2CellsIntersect doesn't match: passed 3, should be 2: In scope SELECT s2CellsIntersect([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:47.353699 [ 2707 ] {205fa5c0-1adb-4a0e-8d88-e0dbefb9cc66} executeQuery: Code: 42. DB::Exception: Number of arguments for function s2GetNeighbors doesn't match: passed 3, should be 1: In scope SELECT s2GetNeighbors([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT s2GetNeighbors([], [], []);) 2025.09.25 21:11:47.359745 [ 2707 ] {205fa5c0-1adb-4a0e-8d88-e0dbefb9cc66} TCPHandler: Code: 42. DB::Exception: Number of arguments for function s2GetNeighbors doesn't match: passed 3, should be 1: In scope SELECT s2GetNeighbors([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:47.444401 [ 2707 ] {e5500b5a-cfd5-465f-bc38-130b6c856bfb} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument 0 of function s2RectAdd. Must be UInt64: In scope SELECT s2RectAdd([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT s2RectAdd([], [], []);) 2025.09.25 21:11:47.457772 [ 2707 ] {e5500b5a-cfd5-465f-bc38-130b6c856bfb} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument 0 of function s2RectAdd. Must be UInt64: In scope SELECT s2RectAdd([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:47.568100 [ 2707 ] {8b0d5966-97ee-4a03-b4ce-44c35334987a} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument 0 of function s2RectContains. Must be UInt64: In scope SELECT s2RectContains([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT s2RectContains([], [], []);) 2025.09.25 21:11:47.574640 [ 2707 ] {8b0d5966-97ee-4a03-b4ce-44c35334987a} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument 0 of function s2RectContains. Must be UInt64: In scope SELECT s2RectContains([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:47.640191 [ 2707 ] {fed42178-5ea7-4555-a2bc-3c7510ad3a5a} executeQuery: Code: 42. DB::Exception: Number of arguments for function s2RectIntersection doesn't match: passed 3, should be 4: In scope SELECT s2RectIntersection([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT s2RectIntersection([], [], []);) 2025.09.25 21:11:47.652376 [ 2707 ] {fed42178-5ea7-4555-a2bc-3c7510ad3a5a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function s2RectIntersection doesn't match: passed 3, should be 4: In scope SELECT s2RectIntersection([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:47.738778 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:47.763713 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:47.786430 [ 2707 ] {dea39307-74cb-4312-930c-5c1aece023df} executeQuery: Code: 42. DB::Exception: Number of arguments for function s2RectUnion doesn't match: passed 3, should be 4: In scope SELECT s2RectUnion([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT s2RectUnion([], [], []);) 2025.09.25 21:11:47.792251 [ 2707 ] {dea39307-74cb-4312-930c-5c1aece023df} TCPHandler: Code: 42. DB::Exception: Number of arguments for function s2RectUnion doesn't match: passed 3, should be 4: In scope SELECT s2RectUnion([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:47.904764 [ 2707 ] {a601f624-579c-4b38-b55c-cd2a49bb2e8a} executeQuery: Code: 42. DB::Exception: Number of arguments for function s2ToGeo doesn't match: passed 3, should be 1: In scope SELECT s2ToGeo([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT s2ToGeo([], [], []);) 2025.09.25 21:11:47.909849 [ 2707 ] {a601f624-579c-4b38-b55c-cd2a49bb2e8a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function s2ToGeo doesn't match: passed 3, should be 1: In scope SELECT s2ToGeo([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:47.952305 [ 2707 ] {a013b557-4e46-4be4-bb6b-2c02a8bbeb05} executeQuery: Code: 42. DB::Exception: Number of arguments for function dotProduct doesn't match: passed 3, should be 2: In scope SELECT scalarProduct([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT scalarProduct([], [], []);) 2025.09.25 21:11:47.956444 [ 2707 ] {a013b557-4e46-4be4-bb6b-2c02a8bbeb05} TCPHandler: Code: 42. DB::Exception: Number of arguments for function dotProduct doesn't match: passed 3, should be 2: In scope SELECT scalarProduct([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:48.099828 [ 2707 ] {768bb3e8-4e18-410b-b464-8ee7eb5f8931} executeQuery: Code: 42. DB::Exception: Aggregate function sequenceCount requires exactly one parameter. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sequenceCount([], [], []);) 2025.09.25 21:11:48.103674 [ 2707 ] {768bb3e8-4e18-410b-b464-8ee7eb5f8931} TCPHandler: Code: 42. DB::Exception: Aggregate function sequenceCount requires exactly one parameter. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:48.154067 [ 2707 ] {38d8ad25-5976-4541-8776-83d4e3b46273} executeQuery: Code: 42. DB::Exception: Aggregate function sequenceMatch requires exactly one parameter. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sequenceMatch([], [], []);) 2025.09.25 21:11:48.157217 [ 2707 ] {38d8ad25-5976-4541-8776-83d4e3b46273} TCPHandler: Code: 42. DB::Exception: Aggregate function sequenceMatch requires exactly one parameter. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:48.177369 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:48.196399 [ 2707 ] {4761831a-2fd9-4457-be1c-b8db409e9978} executeQuery: Code: 63. DB::Exception: Aggregate function sequenceNextNode is experimental. Set `allow_experimental_funnel_functions` setting to enable it. (UNKNOWN_AGGREGATE_FUNCTION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sequenceNextNode([], [], []);) 2025.09.25 21:11:48.197729 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:48.200169 [ 2707 ] {4761831a-2fd9-4457-be1c-b8db409e9978} TCPHandler: Code: 63. DB::Exception: Aggregate function sequenceNextNode is experimental. Set `allow_experimental_funnel_functions` setting to enable it. (UNKNOWN_AGGREGATE_FUNCTION) 2025.09.25 21:11:48.235265 [ 2707 ] {d11895b3-4b8d-41d9-91fc-8977009f6297} executeQuery: Code: 42. DB::Exception: Number of arguments for function seriesDecomposeSTL doesn't match: passed 3, should be 2: In scope SELECT seriesDecomposeSTL([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT seriesDecomposeSTL([], [], []);) 2025.09.25 21:11:48.241263 [ 2707 ] {d11895b3-4b8d-41d9-91fc-8977009f6297} TCPHandler: Code: 42. DB::Exception: Number of arguments for function seriesDecomposeSTL doesn't match: passed 3, should be 2: In scope SELECT seriesDecomposeSTL([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:48.279038 [ 2707 ] {50ec0c05-e74d-4bb8-996b-4c87f5a9e9ff} executeQuery: Code: 42. DB::Exception: Function seriesOutliersDetectTukey needs either 1 or 4 arguments; passed 3.: In scope SELECT seriesOutliersDetectTukey([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT seriesOutliersDetectTukey([], [], []);) 2025.09.25 21:11:48.284513 [ 2707 ] {50ec0c05-e74d-4bb8-996b-4c87f5a9e9ff} TCPHandler: Code: 42. DB::Exception: Function seriesOutliersDetectTukey needs either 1 or 4 arguments; passed 3.: In scope SELECT seriesOutliersDetectTukey([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:48.327246 [ 2707 ] {9b76b08e-7cc7-44ac-bac8-4ba5f8671a01} executeQuery: Code: 42. DB::Exception: Number of arguments for function seriesPeriodDetectFFT doesn't match: passed 3, should be 1: In scope SELECT seriesPeriodDetectFFT([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT seriesPeriodDetectFFT([], [], []);) 2025.09.25 21:11:48.338515 [ 2707 ] {9b76b08e-7cc7-44ac-bac8-4ba5f8671a01} TCPHandler: Code: 42. DB::Exception: Number of arguments for function seriesPeriodDetectFFT doesn't match: passed 3, should be 1: In scope SELECT seriesPeriodDetectFFT([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:48.384924 [ 2707 ] {3f5b0547-088e-4c68-b691-8949871301db} executeQuery: Code: 42. DB::Exception: Number of arguments for function serverTimezone doesn't match: passed 3, should be 0: In scope SELECT serverTimeZone([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT serverTimeZone([], [], []);) 2025.09.25 21:11:48.390044 [ 2707 ] {3f5b0547-088e-4c68-b691-8949871301db} TCPHandler: Code: 42. DB::Exception: Number of arguments for function serverTimezone doesn't match: passed 3, should be 0: In scope SELECT serverTimeZone([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:48.445554 [ 2707 ] {fe8bed02-181c-43e5-919f-dab7fcdb34e3} executeQuery: Code: 42. DB::Exception: Number of arguments for function serverTimezone doesn't match: passed 3, should be 0: In scope SELECT serverTimezone([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT serverTimezone([], [], []);) 2025.09.25 21:11:48.450218 [ 2707 ] {fe8bed02-181c-43e5-919f-dab7fcdb34e3} TCPHandler: Code: 42. DB::Exception: Number of arguments for function serverTimezone doesn't match: passed 3, should be 0: In scope SELECT serverTimezone([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:48.455449 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:48.466680 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:48.483441 [ 2707 ] {4eab34ac-bda3-4339-8eae-449b6edd2499} executeQuery: Code: 42. DB::Exception: Number of arguments for function serverUUID doesn't match: passed 3, should be 0: In scope SELECT serverUUID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT serverUUID([], [], []);) 2025.09.25 21:11:48.487810 [ 2707 ] {4eab34ac-bda3-4339-8eae-449b6edd2499} TCPHandler: Code: 42. DB::Exception: Number of arguments for function serverUUID doesn't match: passed 3, should be 0: In scope SELECT serverUUID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:48.528974 [ 2707 ] {102e8cd7-02a6-410b-861f-395bc5fa9bdf} executeQuery: Code: 42. DB::Exception: Number of arguments for function shardCount doesn't match: passed 3, should be 0: In scope SELECT shardCount([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT shardCount([], [], []);) 2025.09.25 21:11:48.533359 [ 2707 ] {102e8cd7-02a6-410b-861f-395bc5fa9bdf} TCPHandler: Code: 42. DB::Exception: Number of arguments for function shardCount doesn't match: passed 3, should be 0: In scope SELECT shardCount([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:48.580970 [ 2707 ] {e8243bc6-ec9a-407c-8b3f-476d31b41d0d} executeQuery: Code: 42. DB::Exception: Number of arguments for function shardNum doesn't match: passed 3, should be 0: In scope SELECT shardNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT shardNum([], [], []);) 2025.09.25 21:11:48.584992 [ 2707 ] {e8243bc6-ec9a-407c-8b3f-476d31b41d0d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function shardNum doesn't match: passed 3, should be 0: In scope SELECT shardNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:48.638647 [ 2707 ] {dab4d191-c841-46ae-a842-42b68bb0306f} executeQuery: Code: 42. DB::Exception: Number of arguments for function showCertificate doesn't match: passed 3, should be 0: In scope SELECT showCertificate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT showCertificate([], [], []);) 2025.09.25 21:11:48.642491 [ 2707 ] {dab4d191-c841-46ae-a842-42b68bb0306f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function showCertificate doesn't match: passed 3, should be 0: In scope SELECT showCertificate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:48.686062 [ 2707 ] {31c4e11a-9a68-45fb-9286-4008816761e1} executeQuery: Code: 42. DB::Exception: Number of arguments for function sigmoid doesn't match: passed 3, should be 1: In scope SELECT sigmoid([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sigmoid([], [], []);) 2025.09.25 21:11:48.701068 [ 2707 ] {31c4e11a-9a68-45fb-9286-4008816761e1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function sigmoid doesn't match: passed 3, should be 1: In scope SELECT sigmoid([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:48.823247 [ 2707 ] {488ef7da-4873-4557-8f90-43d227492ee0} executeQuery: Code: 42. DB::Exception: Number of arguments for function sign doesn't match: passed 3, should be 1: In scope SELECT sign([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sign([], [], []);) 2025.09.25 21:11:48.829391 [ 2707 ] {488ef7da-4873-4557-8f90-43d227492ee0} TCPHandler: Code: 42. DB::Exception: Number of arguments for function sign doesn't match: passed 3, should be 1: In scope SELECT sign([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:48.879631 [ 2707 ] {9c60ea6d-4318-401f-aac2-02f54b54676a} executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractBool doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractBool([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT simpleJSONExtractBool([], [], []);) 2025.09.25 21:11:48.885784 [ 2707 ] {9c60ea6d-4318-401f-aac2-02f54b54676a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractBool doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractBool([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:48.949805 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:48.961114 [ 2707 ] {a745edfa-87a7-4209-8be0-752e5482dcaa} executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractFloat doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractFloat([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT simpleJSONExtractFloat([], [], []);) 2025.09.25 21:11:48.961168 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:48.970003 [ 2707 ] {a745edfa-87a7-4209-8be0-752e5482dcaa} TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractFloat doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractFloat([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:49.014506 [ 2707 ] {8ae7942c-587f-4e1e-8e89-7d604aa8c928} executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractInt doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractInt([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT simpleJSONExtractInt([], [], []);) 2025.09.25 21:11:49.055001 [ 2707 ] {8ae7942c-587f-4e1e-8e89-7d604aa8c928} TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractInt doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractInt([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:49.101980 [ 2707 ] {bf684b3e-25dd-4e1a-8263-85872d6e352b} executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractRaw doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractRaw([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT simpleJSONExtractRaw([], [], []);) 2025.09.25 21:11:49.106228 [ 2707 ] {bf684b3e-25dd-4e1a-8263-85872d6e352b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractRaw doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractRaw([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:49.163134 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:49.177399 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:49.246675 [ 2707 ] {59c0a526-e27f-4fff-9790-80571ac1abc1} executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractString doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT simpleJSONExtractString([], [], []);) 2025.09.25 21:11:49.249320 [ 2707 ] {59c0a526-e27f-4fff-9790-80571ac1abc1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractString doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:49.282333 [ 2707 ] {0f603d0c-8d81-41a9-a9c0-c73625db342e} executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractUInt doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractUInt([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT simpleJSONExtractUInt([], [], []);) 2025.09.25 21:11:49.288277 [ 2707 ] {0f603d0c-8d81-41a9-a9c0-c73625db342e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractUInt doesn't match: passed 3, should be 2: In scope SELECT simpleJSONExtractUInt([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:49.323519 [ 2707 ] {fbb6d02e-1425-42bf-87cf-1ade6939f41f} executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONHas doesn't match: passed 3, should be 2: In scope SELECT simpleJSONHas([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT simpleJSONHas([], [], []);) 2025.09.25 21:11:49.327175 [ 2707 ] {fbb6d02e-1425-42bf-87cf-1ade6939f41f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONHas doesn't match: passed 3, should be 2: In scope SELECT simpleJSONHas([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:49.531968 [ 2707 ] {c9cc2b58-afa8-47ac-940b-e574c5f3f0ee} executeQuery: Code: 42. DB::Exception: Aggregate function simpleLinearRegression requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT simpleLinearRegression([], [], []);) 2025.09.25 21:11:49.535086 [ 2707 ] {c9cc2b58-afa8-47ac-940b-e574c5f3f0ee} TCPHandler: Code: 42. DB::Exception: Aggregate function simpleLinearRegression requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:49.594832 [ 2707 ] {c84cd49f-547a-4e9b-953f-5b872984e490} executeQuery: Code: 42. DB::Exception: Number of arguments for function sin doesn't match: passed 3, should be 1: In scope SELECT sin([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sin([], [], []);) 2025.09.25 21:11:49.601970 [ 2707 ] {c84cd49f-547a-4e9b-953f-5b872984e490} TCPHandler: Code: 42. DB::Exception: Number of arguments for function sin doesn't match: passed 3, should be 1: In scope SELECT sin([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:49.774159 [ 2707 ] {ee94d69b-b80e-4d0f-8990-adead99f09e0} executeQuery: Code: 42. DB::Exception: Aggregate function singleValueOrNull requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT singleValueOrNull([], [], []);) 2025.09.25 21:11:49.781541 [ 2707 ] {ee94d69b-b80e-4d0f-8990-adead99f09e0} TCPHandler: Code: 42. DB::Exception: Aggregate function singleValueOrNull requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:49.832538 [ 2707 ] {877a6b8a-6376-4a3d-a114-d856edd886a8} executeQuery: Code: 42. DB::Exception: Number of arguments for function sinh doesn't match: passed 3, should be 1: In scope SELECT sinh([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sinh([], [], []);) 2025.09.25 21:11:49.847960 [ 2707 ] {877a6b8a-6376-4a3d-a114-d856edd886a8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function sinh doesn't match: passed 3, should be 1: In scope SELECT sinh([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:50.015987 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:50.016773 [ 2707 ] {796a17ff-8879-4ca0-9166-be0dfb9b5fe3} executeQuery: Code: 36. DB::Exception: The key must be of type Tuple(UInt64, UInt64): In scope SELECT sipHash128Keyed([], [], []). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sipHash128Keyed([], [], []);) 2025.09.25 21:11:50.023945 [ 2707 ] {796a17ff-8879-4ca0-9166-be0dfb9b5fe3} TCPHandler: Code: 36. DB::Exception: The key must be of type Tuple(UInt64, UInt64): In scope SELECT sipHash128Keyed([], [], []). (BAD_ARGUMENTS) 2025.09.25 21:11:50.033092 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:50.204148 [ 2707 ] {3ab6c6c6-6f74-448d-a1a2-f609a3d8d5a6} executeQuery: Code: 36. DB::Exception: The key must be of type Tuple(UInt64, UInt64): In scope SELECT sipHash128ReferenceKeyed([], [], []). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sipHash128ReferenceKeyed([], [], []);) 2025.09.25 21:11:50.209342 [ 2707 ] {3ab6c6c6-6f74-448d-a1a2-f609a3d8d5a6} TCPHandler: Code: 36. DB::Exception: The key must be of type Tuple(UInt64, UInt64): In scope SELECT sipHash128ReferenceKeyed([], [], []). (BAD_ARGUMENTS) 2025.09.25 21:11:50.377003 [ 2707 ] {60d757e0-a459-4e63-90cd-befaf3f11ca7} executeQuery: Code: 36. DB::Exception: The key must be of type Tuple(UInt64, UInt64): In scope SELECT sipHash64Keyed([], [], []). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sipHash64Keyed([], [], []);) 2025.09.25 21:11:50.383995 [ 2707 ] {60d757e0-a459-4e63-90cd-befaf3f11ca7} TCPHandler: Code: 36. DB::Exception: The key must be of type Tuple(UInt64, UInt64): In scope SELECT sipHash64Keyed([], [], []). (BAD_ARGUMENTS) 2025.09.25 21:11:50.471728 [ 2707 ] {eca713c8-8f2b-42df-90f7-e10eab38c18f} executeQuery: Code: 42. DB::Exception: Aggregate function skewPop requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT skewPop([], [], []);) 2025.09.25 21:11:50.479379 [ 2707 ] {eca713c8-8f2b-42df-90f7-e10eab38c18f} TCPHandler: Code: 42. DB::Exception: Aggregate function skewPop requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:50.525620 [ 2707 ] {a6d95744-011e-4033-9511-718d2b0c1948} executeQuery: Code: 42. DB::Exception: Aggregate function skewSamp requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT skewSamp([], [], []);) 2025.09.25 21:11:50.535676 [ 2707 ] {a6d95744-011e-4033-9511-718d2b0c1948} TCPHandler: Code: 42. DB::Exception: Aggregate function skewSamp requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:50.601061 [ 2707 ] {92549605-4e94-4525-88a1-9683999605c3} executeQuery: Code: 42. DB::Exception: Number of arguments for function sleep doesn't match: passed 3, should be 1: In scope SELECT sleep([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sleep([], [], []);) 2025.09.25 21:11:50.605979 [ 2707 ] {92549605-4e94-4525-88a1-9683999605c3} TCPHandler: Code: 42. DB::Exception: Number of arguments for function sleep doesn't match: passed 3, should be 1: In scope SELECT sleep([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:50.654814 [ 2707 ] {3f9c70fb-d338-4c0b-98b9-b3ff30c62051} executeQuery: Code: 42. DB::Exception: Number of arguments for function sleepEachRow doesn't match: passed 3, should be 1: In scope SELECT sleepEachRow([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sleepEachRow([], [], []);) 2025.09.25 21:11:50.727333 [ 2707 ] {3f9c70fb-d338-4c0b-98b9-b3ff30c62051} TCPHandler: Code: 42. DB::Exception: Number of arguments for function sleepEachRow doesn't match: passed 3, should be 1: In scope SELECT sleepEachRow([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:50.819505 [ 2707 ] {31bc01ef-f83e-428f-8ef3-915fcd82078f} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'value' to function 'snowflakeIDToDateTime'. Expected: UInt64, got: Array(Nothing): In scope SELECT snowflakeIDToDateTime([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT snowflakeIDToDateTime([], [], []);) 2025.09.25 21:11:50.823041 [ 2707 ] {31bc01ef-f83e-428f-8ef3-915fcd82078f} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'value' to function 'snowflakeIDToDateTime'. Expected: UInt64, got: Array(Nothing): In scope SELECT snowflakeIDToDateTime([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:50.904135 [ 2707 ] {622376b1-72de-45cb-9d50-7c74b3c447ba} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'value' to function 'snowflakeIDToDateTime64'. Expected: UInt64, got: Array(Nothing): In scope SELECT snowflakeIDToDateTime64([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT snowflakeIDToDateTime64([], [], []);) 2025.09.25 21:11:50.909031 [ 2707 ] {622376b1-72de-45cb-9d50-7c74b3c447ba} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'value' to function 'snowflakeIDToDateTime64'. Expected: UInt64, got: Array(Nothing): In scope SELECT snowflakeIDToDateTime64([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:51.000863 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:51.019900 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:51.053443 [ 2707 ] {58ac3be6-b7e2-4108-a6a9-021014450ccb} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'snowflakeToDateTime'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT snowflakeToDateTime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT snowflakeToDateTime([], [], []);) 2025.09.25 21:11:51.059943 [ 2707 ] {58ac3be6-b7e2-4108-a6a9-021014450ccb} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'snowflakeToDateTime'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT snowflakeToDateTime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:51.125814 [ 2707 ] {a2e5d576-588b-493c-902d-501e8a3efa1d} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'snowflakeToDateTime64'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT snowflakeToDateTime64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT snowflakeToDateTime64([], [], []);) 2025.09.25 21:11:51.140100 [ 2707 ] {a2e5d576-588b-493c-902d-501e8a3efa1d} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'snowflakeToDateTime64'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT snowflakeToDateTime64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:51.200490 [ 2707 ] {df18cdcb-9c41-4307-b96b-4f37f404cf13} executeQuery: Code: 42. DB::Exception: Number of arguments for function soundex doesn't match: passed 3, should be 1: In scope SELECT soundex([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT soundex([], [], []);) 2025.09.25 21:11:51.220941 [ 2707 ] {df18cdcb-9c41-4307-b96b-4f37f404cf13} TCPHandler: Code: 42. DB::Exception: Number of arguments for function soundex doesn't match: passed 3, should be 1: In scope SELECT soundex([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:51.292435 [ 2707 ] {111cadd2-1560-4b2d-bf57-31646269e3af} executeQuery: Code: 42. DB::Exception: Number of arguments for function space doesn't match: passed 3, should be 1: In scope SELECT space([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT space([], [], []);) 2025.09.25 21:11:51.298036 [ 2707 ] {111cadd2-1560-4b2d-bf57-31646269e3af} TCPHandler: Code: 42. DB::Exception: Number of arguments for function space doesn't match: passed 3, should be 1: In scope SELECT space([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:51.355269 [ 2707 ] {879d7d5b-feb4-472d-aeb0-6aff6de31115} executeQuery: Code: 42. DB::Exception: Aggregate function sparkbar requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sparkBar([], [], []);) 2025.09.25 21:11:51.359858 [ 2707 ] {879d7d5b-feb4-472d-aeb0-6aff6de31115} TCPHandler: Code: 42. DB::Exception: Aggregate function sparkbar requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:51.449837 [ 2707 ] {45cb7fcd-7470-4e9d-9154-451406b3e3a6} executeQuery: Code: 42. DB::Exception: Aggregate function sparkbar requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sparkbar([], [], []);) 2025.09.25 21:11:51.457523 [ 2707 ] {45cb7fcd-7470-4e9d-9154-451406b3e3a6} TCPHandler: Code: 42. DB::Exception: Aggregate function sparkbar requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:51.481832 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:51.550778 [ 2707 ] {999460d6-558e-430b-b97c-4a5baa1f98d5} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'alphaTokens'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT splitByAlpha([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT splitByAlpha([], [], []);) 2025.09.25 21:11:51.570786 [ 2707 ] {999460d6-558e-430b-b97c-4a5baa1f98d5} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'alphaTokens'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT splitByAlpha([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:51.587307 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:51.640943 [ 2707 ] {63269114-a85a-42ca-816d-eda857223b19} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'separator' to function 'splitByChar'. Expected: const String, got: Array(Nothing): In scope SELECT splitByChar([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT splitByChar([], [], []);) 2025.09.25 21:11:51.647823 [ 2707 ] {63269114-a85a-42ca-816d-eda857223b19} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'separator' to function 'splitByChar'. Expected: const String, got: Array(Nothing): In scope SELECT splitByChar([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:51.719768 [ 2707 ] {d0fb52d7-2496-43b9-85f2-028fc236ce61} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'splitByNonAlpha'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT splitByNonAlpha([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT splitByNonAlpha([], [], []);) 2025.09.25 21:11:51.729635 [ 2707 ] {d0fb52d7-2496-43b9-85f2-028fc236ce61} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'splitByNonAlpha'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT splitByNonAlpha([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:51.828096 [ 3851 ] {251abee5-a533-4dbb-9fca-2975a7a91c97} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58262) (comment: 00506_union_distributed.sql) (in query: SELECT `__table1`.`b` AS `b`, sum(`__table1`.`c`) AS `c` FROM `test_sz5whowb`.`union1` AS `__table1` WHERE `__table1`.`a` > 1 GROUP BY `__table1`.`a`, `__table1`.`b`), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:11:51.835223 [ 3851 ] {251abee5-a533-4dbb-9fca-2975a7a91c97} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:11:51.873158 [ 2707 ] {ae3efdee-740e-4c27-b6f7-d313cfd222b6} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'separator' to function 'splitByRegexp'. Expected: const String, got: Array(Nothing): In scope SELECT splitByRegexp([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT splitByRegexp([], [], []);) 2025.09.25 21:11:51.884179 [ 2707 ] {ae3efdee-740e-4c27-b6f7-d313cfd222b6} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'separator' to function 'splitByRegexp'. Expected: const String, got: Array(Nothing): In scope SELECT splitByRegexp([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:51.908176 [ 7168 ] {d548d501-5250-4da8-9c20-34d0f3587eb9} executeQuery: Code: 439. DB::Exception: Received from localhost:9000. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). Stack trace: 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 : While executing Remote. (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50446) (comment: 00506_union_distributed.sql) (in query: select b, sum(c) from ( select a, b, sum(c) as c from union2 where a>1 group by a,b UNION ALL select a, b, sum(c) as c from union3 where b>1 group by a, b order by a, b) as a group by b order by b;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001aeb974e 3. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000002cee6c6a 4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x000000002cef780e 5. ./build_docker/./src/Client/PacketReceiver.cpp:0: DB::PacketReceiver::Task::run(std::function, std::function) @ 0x000000002cf9767e 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278ab436 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278aaa44 8. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func>>(unsigned int, unsigned int) @ 0x00000000278a8394 2025.09.25 21:11:51.913476 [ 7168 ] {d548d501-5250-4da8-9c20-34d0f3587eb9} TCPHandler: Code: 439. DB::Exception: Received from localhost:9000. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). Stack trace: 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 : While executing Remote. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001aeb974e 3. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000002cee6c6a 4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x000000002cef780e 5. ./build_docker/./src/Client/PacketReceiver.cpp:0: DB::PacketReceiver::Task::run(std::function, std::function) @ 0x000000002cf9767e 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278ab436 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278aaa44 8. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func>>(unsigned int, unsigned int) @ 0x00000000278a8394 2025.09.25 21:11:51.970745 [ 2707 ] {f1c524dc-b82e-408c-a539-5185f9a3888d} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'separator' to function 'splitByString'. Expected: const String, got: Array(Nothing): In scope SELECT splitByString([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT splitByString([], [], []);) 2025.09.25 21:11:51.989209 [ 2707 ] {f1c524dc-b82e-408c-a539-5185f9a3888d} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'separator' to function 'splitByString'. Expected: const String, got: Array(Nothing): In scope SELECT splitByString([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:52.044208 [ 3682 ] {e9ddc00b-6557-4ff5-9c31-51c4ee9a4f85} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::1]:9000 -> [::1]:50452). (NETWORK_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50452) (comment: 00506_union_distributed.sql) (in query: SELECT `__table1`.`b` AS `b`, sum(`__table1`.`c`) AS `c` FROM `test_sz5whowb`.`union2` AS `__table1` WHERE `__table1`.`b` > 1 GROUP BY `__table1`.`a`, `__table1`.`b` ORDER BY `__table1`.`a` ASC, `__table1`.`b` ASC), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001b0bab02 4. ./build_docker/./src/IO/WriteBufferFromPocoSocketChunked.cpp:201: DB::WriteBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf09bf7 5. ./src/IO/WriteBuffer.h:66: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000002d190f6b 6. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18b989 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:11:52.051137 [ 3682 ] {e9ddc00b-6557-4ff5-9c31-51c4ee9a4f85} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::1]:9000 -> [::1]:50452). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001b0bada2 4. ./build_docker/./src/IO/WriteBufferFromPocoSocketChunked.cpp:201: DB::WriteBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf09bf7 5. ./src/IO/WriteBuffer.h:66: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000002d1a9beb 6. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x000000002d18db59 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d176902 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:52.052784 [ 3682 ] {e9ddc00b-6557-4ff5-9c31-51c4ee9a4f85} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::1]:9000 -> [::1]:50452). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001b0bab02 4. ./build_docker/./src/IO/WriteBufferFromPocoSocketChunked.cpp:201: DB::WriteBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf09bf7 5. ./src/IO/WriteBuffer.h:66: DB::TCPHandler::sendData(DB::Block const&) @ 0x000000002d190f6b 6. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18b989 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:11:52.060038 [ 3682 ] {e9ddc00b-6557-4ff5-9c31-51c4ee9a4f85} TCPHandler: Client has gone away. 2025.09.25 21:11:52.071130 [ 2707 ] {333d6c38-e6ac-49fd-823f-84db5476ef75} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'splitByWhitespace'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT splitByWhitespace([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT splitByWhitespace([], [], []);) 2025.09.25 21:11:52.077496 [ 2707 ] {333d6c38-e6ac-49fd-823f-84db5476ef75} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'splitByWhitespace'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT splitByWhitespace([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:52.114029 [ 3352 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {f2eeeeb5-70de-4a33-835e-3bff489e4be8::202509_1_44_9}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:52.153116 [ 3357 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {f2eeeeb5-70de-4a33-835e-3bff489e4be8::202509_1_44_9}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:52.176399 [ 2707 ] {3868054c-b20e-4321-aaac-1daebc55d8bf} executeQuery: Code: 43. DB::Exception: Argument 0 for function sqidEncode must have datatype UInt*, given type: Array(Nothing).: In scope SELECT sqid([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sqid([], [], []);) 2025.09.25 21:11:52.182307 [ 2707 ] {3868054c-b20e-4321-aaac-1daebc55d8bf} TCPHandler: Code: 43. DB::Exception: Argument 0 for function sqidEncode must have datatype UInt*, given type: Array(Nothing).: In scope SELECT sqid([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:52.188012 [ 8396 ] {f883c0e8-79f2-4ee4-b05b-30689d7d7d7a} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::1]:9000 -> [::1]:50458). (NETWORK_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50458) (comment: 00506_union_distributed.sql) (in query: SELECT `__table1`.`b` AS `b`, sum(`__table1`.`c`) AS `c` FROM `test_sz5whowb`.`union1` AS `__table1` WHERE `__table1`.`b` > 1 GROUP BY `__table1`.`a`, `__table1`.`b` ORDER BY `__table1`.`a` ASC, `__table1`.`b` ASC), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001b0bada2 4. ./build_docker/./src/IO/WriteBufferFromPocoSocketChunked.cpp:201: DB::WriteBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf09bf7 5. ./src/IO/WriteBuffer.h:66: DB::TCPHandler::sendProgress() @ 0x000000002d18c7b2 6. ./build_docker/./src/Server/TCPHandler.cpp:1170: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18be00 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:11:52.192401 [ 8396 ] {f883c0e8-79f2-4ee4-b05b-30689d7d7d7a} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::1]:9000 -> [::1]:50458). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001b0bada2 4. ./build_docker/./src/IO/WriteBufferFromPocoSocketChunked.cpp:201: DB::WriteBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf09bf7 5. ./src/IO/WriteBuffer.h:66: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000002d1a9beb 6. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x000000002d18db59 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d176902 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:52.194776 [ 8396 ] {f883c0e8-79f2-4ee4-b05b-30689d7d7d7a} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::1]:9000 -> [::1]:50458). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001b0bada2 4. ./build_docker/./src/IO/WriteBufferFromPocoSocketChunked.cpp:201: DB::WriteBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf09bf7 5. ./src/IO/WriteBuffer.h:66: DB::TCPHandler::sendProgress() @ 0x000000002d18c7b2 6. ./build_docker/./src/Server/TCPHandler.cpp:1170: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18be00 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:11:52.196777 [ 8396 ] {f883c0e8-79f2-4ee4-b05b-30689d7d7d7a} TCPHandler: Client has gone away. 2025.09.25 21:11:52.236362 [ 2707 ] {b1c655e1-7c97-40d6-9a44-26806d2863dc} executeQuery: Code: 42. DB::Exception: Number of arguments for function sqidDecode doesn't match: passed 3, should be 1: In scope SELECT sqidDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sqidDecode([], [], []);) 2025.09.25 21:11:52.239518 [ 2707 ] {b1c655e1-7c97-40d6-9a44-26806d2863dc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function sqidDecode doesn't match: passed 3, should be 1: In scope SELECT sqidDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:52.296050 [ 3619 ] {ce1c8a83-0408-400f-9369-1fcd3b9a06f2} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:44382) (comment: 00506_union_distributed.sql) (in query: DROP DATABASE IF EXISTS test_sz5whowb ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:11:52.299112 [ 3619 ] {ce1c8a83-0408-400f-9369-1fcd3b9a06f2} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:52.351863 [ 2707 ] {42ca97c0-ee94-40fb-b3b0-9bb0b26401a5} executeQuery: Code: 43. DB::Exception: Argument 0 for function sqidEncode must have datatype UInt*, given type: Array(Nothing).: In scope SELECT sqidEncode([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sqidEncode([], [], []);) 2025.09.25 21:11:52.365356 [ 2707 ] {42ca97c0-ee94-40fb-b3b0-9bb0b26401a5} TCPHandler: Code: 43. DB::Exception: Argument 0 for function sqidEncode must have datatype UInt*, given type: Array(Nothing).: In scope SELECT sqidEncode([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:52.377806 [ 2938 ] {} DatabaseCatalog: Cannot drop tables. Will retry later.: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000286c6b9c 8. ./contrib/llvm-project/libcxx/include/vector:590: DB::DatabaseCatalog::dropTableDataTask() @ 0x00000000286c7466 9. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 11. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 14. asan_thread_start(void*) @ 0x000000000aa43059 15. ? @ 0x00007f12cd75eac3 16. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:52.404723 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:52.413342 [ 2707 ] {06c80f7d-84fb-434f-a942-7ab800e7b8e7} executeQuery: Code: 42. DB::Exception: Number of arguments for function sqrt doesn't match: passed 3, should be 1: In scope SELECT sqrt([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sqrt([], [], []);) 2025.09.25 21:11:52.415471 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:52.417775 [ 2707 ] {06c80f7d-84fb-434f-a942-7ab800e7b8e7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function sqrt doesn't match: passed 3, should be 1: In scope SELECT sqrt([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:52.461653 [ 2707 ] {d64a3e51-1a5d-47b4-8a8a-53a4e81229ed} executeQuery: Code: 42. DB::Exception: Number of arguments for function startsWith doesn't match: passed 3, should be 2: In scope SELECT startsWith([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT startsWith([], [], []);) 2025.09.25 21:11:52.466449 [ 2707 ] {d64a3e51-1a5d-47b4-8a8a-53a4e81229ed} TCPHandler: Code: 42. DB::Exception: Number of arguments for function startsWith doesn't match: passed 3, should be 2: In scope SELECT startsWith([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:52.550675 [ 2707 ] {a2fa6dfb-b2ac-4d5b-84e3-4c984742be2e} executeQuery: Code: 42. DB::Exception: Number of arguments for function startsWithUTF8 doesn't match: passed 3, should be 2: In scope SELECT startsWithUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT startsWithUTF8([], [], []);) 2025.09.25 21:11:52.557830 [ 2707 ] {a2fa6dfb-b2ac-4d5b-84e3-4c984742be2e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function startsWithUTF8 doesn't match: passed 3, should be 2: In scope SELECT startsWithUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:52.618616 [ 2707 ] {f18f2a54-8410-42f1-9b42-8b2555ddea16} executeQuery: Code: 42. DB::Exception: Aggregate function stddevPop requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT stddevPop([], [], []);) 2025.09.25 21:11:52.624541 [ 2707 ] {f18f2a54-8410-42f1-9b42-8b2555ddea16} TCPHandler: Code: 42. DB::Exception: Aggregate function stddevPop requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:52.693826 [ 2707 ] {cc275b58-18a7-4ac3-9ea4-d8717f66410c} executeQuery: Code: 42. DB::Exception: Aggregate function stddevPopStable requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT stddevPopStable([], [], []);) 2025.09.25 21:11:52.699206 [ 2707 ] {cc275b58-18a7-4ac3-9ea4-d8717f66410c} TCPHandler: Code: 42. DB::Exception: Aggregate function stddevPopStable requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:52.765042 [ 2707 ] {42ebc578-920e-41f6-80ff-f3d20378dc84} executeQuery: Code: 42. DB::Exception: Aggregate function stddevSamp requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT stddevSamp([], [], []);) 2025.09.25 21:11:52.769884 [ 2707 ] {42ebc578-920e-41f6-80ff-f3d20378dc84} TCPHandler: Code: 42. DB::Exception: Aggregate function stddevSamp requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:52.874527 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:52.890354 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:52.922877 [ 2707 ] {c06fed5a-7872-4553-a00b-16adb0003179} executeQuery: Code: 42. DB::Exception: Aggregate function stddevSampStable requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT stddevSampStable([], [], []);) 2025.09.25 21:11:52.937845 [ 2707 ] {c06fed5a-7872-4553-a00b-16adb0003179} TCPHandler: Code: 42. DB::Exception: Aggregate function stddevSampStable requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:53.029812 [ 2707 ] {826c4de5-3f63-450f-9503-363215f72138} executeQuery: Code: 344. DB::Exception: Natural language processing function 'stem' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT stem([], [], []);) 2025.09.25 21:11:53.043220 [ 2707 ] {826c4de5-3f63-450f-9503-363215f72138} TCPHandler: Code: 344. DB::Exception: Natural language processing function 'stem' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) 2025.09.25 21:11:53.086311 [ 2707 ] {8163f6de-ac2e-4a71-a0a7-4d71f092b018} executeQuery: Code: 43. DB::Exception: Argument 0 of type Array(Nothing) must be numeric for aggregate function stochasticLinearRegression. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT stochasticLinearRegression([], [], []);) 2025.09.25 21:11:53.089915 [ 2707 ] {8163f6de-ac2e-4a71-a0a7-4d71f092b018} TCPHandler: Code: 43. DB::Exception: Argument 0 of type Array(Nothing) must be numeric for aggregate function stochasticLinearRegression. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:53.137488 [ 2707 ] {cbb070b1-8551-4254-bef6-0d4e8fe50cb3} executeQuery: Code: 43. DB::Exception: Argument 0 of type Array(Nothing) must be numeric for aggregate function stochasticLogisticRegression. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT stochasticLogisticRegression([], [], []);) 2025.09.25 21:11:53.143243 [ 2707 ] {cbb070b1-8551-4254-bef6-0d4e8fe50cb3} TCPHandler: Code: 43. DB::Exception: Argument 0 of type Array(Nothing) must be numeric for aggregate function stochasticLogisticRegression. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:53.227317 [ 2707 ] {38fe9cf2-8623-4a9a-99fc-f70ba0ff5306} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'time' to function 'parseDateTimeOrNull'. Expected: String, got: Array(Nothing): In scope SELECT str_to_date([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT str_to_date([], [], []);) 2025.09.25 21:11:53.232100 [ 2707 ] {38fe9cf2-8623-4a9a-99fc-f70ba0ff5306} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'time' to function 'parseDateTimeOrNull'. Expected: String, got: Array(Nothing): In scope SELECT str_to_date([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:53.407525 [ 2707 ] {88ed8484-552d-490f-af79-4240450c6d7d} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument data_column. Must be String.: In scope SELECT str_to_map([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT str_to_map([], [], []);) 2025.09.25 21:11:53.411863 [ 2707 ] {88ed8484-552d-490f-af79-4240450c6d7d} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument data_column. Must be String.: In scope SELECT str_to_map([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:53.519875 [ 2707 ] {acedbd95-daaf-4705-afef-c37da99257bc} executeQuery: Code: 42. DB::Exception: Number of arguments for function stringJaccardIndex doesn't match: passed 3, should be 2: In scope SELECT stringJaccardIndex([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT stringJaccardIndex([], [], []);) 2025.09.25 21:11:53.608407 [ 2707 ] {acedbd95-daaf-4705-afef-c37da99257bc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function stringJaccardIndex doesn't match: passed 3, should be 2: In scope SELECT stringJaccardIndex([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:53.661649 [ 2707 ] {086cd0e6-8880-46b9-8cc7-a68289f54cac} executeQuery: Code: 42. DB::Exception: Number of arguments for function stringJaccardIndexUTF8 doesn't match: passed 3, should be 2: In scope SELECT stringJaccardIndexUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT stringJaccardIndexUTF8([], [], []);) 2025.09.25 21:11:53.668632 [ 2707 ] {086cd0e6-8880-46b9-8cc7-a68289f54cac} TCPHandler: Code: 42. DB::Exception: Number of arguments for function stringJaccardIndexUTF8 doesn't match: passed 3, should be 2: In scope SELECT stringJaccardIndexUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:53.727825 [ 2707 ] {9bbf4045-414a-4ede-ac47-4d450ebb601d} executeQuery: Code: 42. DB::Exception: Number of arguments for function stringToH3 doesn't match: passed 3, should be 1: In scope SELECT stringToH3([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT stringToH3([], [], []);) 2025.09.25 21:11:53.732831 [ 2707 ] {9bbf4045-414a-4ede-ac47-4d450ebb601d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function stringToH3 doesn't match: passed 3, should be 1: In scope SELECT stringToH3([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:53.903416 [ 2707 ] {61b470ce-50fb-408b-9d09-4ffe13f31012} executeQuery: Code: 42. DB::Exception: Number of arguments for function structureToCapnProtoSchema doesn't match: passed 3, expected 1 or 2: In scope SELECT structureToCapnProtoSchema([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT structureToCapnProtoSchema([], [], []);) 2025.09.25 21:11:53.907260 [ 2707 ] {61b470ce-50fb-408b-9d09-4ffe13f31012} TCPHandler: Code: 42. DB::Exception: Number of arguments for function structureToCapnProtoSchema doesn't match: passed 3, expected 1 or 2: In scope SELECT structureToCapnProtoSchema([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:53.945513 [ 2707 ] {4ab6cfaa-d0dd-4157-903b-72bdaee30751} executeQuery: Code: 42. DB::Exception: Number of arguments for function structureToProtobufSchema doesn't match: passed 3, expected 1 or 2: In scope SELECT structureToProtobufSchema([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT structureToProtobufSchema([], [], []);) 2025.09.25 21:11:53.948700 [ 2707 ] {4ab6cfaa-d0dd-4157-903b-72bdaee30751} TCPHandler: Code: 42. DB::Exception: Number of arguments for function structureToProtobufSchema doesn't match: passed 3, expected 1 or 2: In scope SELECT structureToProtobufSchema([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:53.987928 [ 2707 ] {b69a5504-4186-4be3-9548-ead1a51b6012} executeQuery: Code: 42. DB::Exception: Aggregate function studentTTest requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT studentTTest([], [], []);) 2025.09.25 21:11:53.991208 [ 2707 ] {b69a5504-4186-4be3-9548-ead1a51b6012} TCPHandler: Code: 42. DB::Exception: Aggregate function studentTTest requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:54.021604 [ 2707 ] {dcd5afab-bccd-4c2b-8ada-8a00d8ff7066} executeQuery: Code: 43. DB::Exception: First argument for function subBitmap must be a bitmap but it has type Array(Nothing).: In scope SELECT subBitmap([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT subBitmap([], [], []);) 2025.09.25 21:11:54.024285 [ 2707 ] {dcd5afab-bccd-4c2b-8ada-8a00d8ff7066} TCPHandler: Code: 43. DB::Exception: First argument for function subBitmap must be a bitmap but it has type Array(Nothing).: In scope SELECT subBitmap([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:54.093448 [ 2707 ] {5316f7de-480d-47c2-9f82-fc9bf6393390} executeQuery: Code: 42. DB::Exception: Number of arguments for function subDate doesn't match: passed 3, should be 2: In scope SELECT subDate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT subDate([], [], []);) 2025.09.25 21:11:54.097209 [ 2707 ] {5316f7de-480d-47c2-9f82-fc9bf6393390} TCPHandler: Code: 42. DB::Exception: Number of arguments for function subDate doesn't match: passed 3, should be 2: In scope SELECT subDate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:54.116085 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:54.153222 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:54.241296 [ 2707 ] {d8f5f0bf-eb87-4397-b4fe-e4a52ac8fb3e} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function substring, expected String, FixedString or Enum: In scope SELECT substr([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT substr([], [], []);) 2025.09.25 21:11:54.245125 [ 2707 ] {d8f5f0bf-eb87-4397-b4fe-e4a52ac8fb3e} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function substring, expected String, FixedString or Enum: In scope SELECT substr([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:54.311464 [ 2707 ] {9a710382-668a-436a-acfd-4b75a690ee14} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function substring, expected String, FixedString or Enum: In scope SELECT substring([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT substring([], [], []);) 2025.09.25 21:11:54.316340 [ 2707 ] {9a710382-668a-436a-acfd-4b75a690ee14} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function substring, expected String, FixedString or Enum: In scope SELECT substring([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:54.539959 [ 2707 ] {b86833d3-e2b1-4814-9ffa-9efd6fe689cb} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function substringIndex, String expected: In scope SELECT substringIndex([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT substringIndex([], [], []);) 2025.09.25 21:11:54.543662 [ 2707 ] {b86833d3-e2b1-4814-9ffa-9efd6fe689cb} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function substringIndex, String expected: In scope SELECT substringIndex([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:54.625986 [ 2707 ] {d9cbeed0-9442-4ecd-9f38-65f071efa0ce} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function substringIndexUTF8, String expected: In scope SELECT substringIndexUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT substringIndexUTF8([], [], []);) 2025.09.25 21:11:54.633159 [ 2707 ] {d9cbeed0-9442-4ecd-9f38-65f071efa0ce} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function substringIndexUTF8, String expected: In scope SELECT substringIndexUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:54.738788 [ 2707 ] {a030c394-773d-4990-9faa-7ca9c539a5d5} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function substringUTF8, expected String: In scope SELECT substringUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT substringUTF8([], [], []);) 2025.09.25 21:11:54.748097 [ 2707 ] {a030c394-773d-4990-9faa-7ca9c539a5d5} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function substringUTF8, expected String: In scope SELECT substringUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:54.844707 [ 2707 ] {97fae46d-c289-457e-832d-15240e74ab1a} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractDays. Must be a DateTime/DateTime64: In scope SELECT subtractDays([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT subtractDays([], [], []);) 2025.09.25 21:11:54.856845 [ 2707 ] {97fae46d-c289-457e-832d-15240e74ab1a} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractDays. Must be a DateTime/DateTime64: In scope SELECT subtractDays([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:54.994333 [ 2707 ] {4e43909b-1f97-404e-8e35-4ea3c9266185} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractHours. Must be a DateTime/DateTime64: In scope SELECT subtractHours([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT subtractHours([], [], []);) 2025.09.25 21:11:55.003522 [ 2707 ] {4e43909b-1f97-404e-8e35-4ea3c9266185} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractHours. Must be a DateTime/DateTime64: In scope SELECT subtractHours([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:55.082234 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:55.088395 [ 2707 ] {8b4068ef-91cf-4dd5-a755-94ddd8a9a0b1} executeQuery: Code: 42. DB::Exception: Number of arguments for function subtractInterval doesn't match: passed 3, should be 2: In scope SELECT subtractInterval([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT subtractInterval([], [], []);) 2025.09.25 21:11:55.094160 [ 2707 ] {8b4068ef-91cf-4dd5-a755-94ddd8a9a0b1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function subtractInterval doesn't match: passed 3, should be 2: In scope SELECT subtractInterval([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:55.098162 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:55.135798 [ 2707 ] {f7e84e5f-5219-4104-8563-b6c6c04b079e} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractMicroseconds. Must be a DateTime/DateTime64: In scope SELECT subtractMicroseconds([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT subtractMicroseconds([], [], []);) 2025.09.25 21:11:55.138923 [ 2707 ] {f7e84e5f-5219-4104-8563-b6c6c04b079e} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractMicroseconds. Must be a DateTime/DateTime64: In scope SELECT subtractMicroseconds([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:55.238558 [ 3619 ] {f41a981d-a891-4990-b542-63a5111a89e1} executeQuery: Code: 42. DB::Exception: Table function 'executable' requires minimum 3 arguments: script_name, format, structure, [input_query...]. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50478) (comment: 03006_analyzer_executable_table_function.sql) (in query: SELECT toFixedString(toFixedString(toLowCardinality(toFixedString('--------------------', toNullable(20))), toLowCardinality(20)), 20), * FROM executable('data String', SETTINGS max_command_execution_time = 100);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/TableFunctions/TableFunctionExecutable.cpp:97: DB::(anonymous namespace)::TableFunctionExecutable::parseArguments(std::shared_ptr const&, std::shared_ptr) @ 0x0000000025fdd6a9 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveTableFunction(std::shared_ptr&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&, bool) @ 0x000000002836cae2 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x00000000283ba404 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ad09 8. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 10. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 13. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 14. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:11:55.241100 [ 3619 ] {f41a981d-a891-4990-b542-63a5111a89e1} TCPHandler: Code: 42. DB::Exception: Table function 'executable' requires minimum 3 arguments: script_name, format, structure, [input_query...]. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/TableFunctions/TableFunctionExecutable.cpp:97: DB::(anonymous namespace)::TableFunctionExecutable::parseArguments(std::shared_ptr const&, std::shared_ptr) @ 0x0000000025fdd6a9 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveTableFunction(std::shared_ptr&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&, bool) @ 0x000000002836cae2 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x00000000283ba404 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ad09 8. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 10. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 13. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 14. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:11:55.286610 [ 2707 ] {b21e13ae-2747-409e-8bf1-2c6f15677be0} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractMilliseconds. Must be a DateTime/DateTime64: In scope SELECT subtractMilliseconds([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT subtractMilliseconds([], [], []);) 2025.09.25 21:11:55.295642 [ 2707 ] {b21e13ae-2747-409e-8bf1-2c6f15677be0} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractMilliseconds. Must be a DateTime/DateTime64: In scope SELECT subtractMilliseconds([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:55.419351 [ 2707 ] {04af0601-6df2-4258-a9aa-02fd9a8aa8bb} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractMinutes. Must be a DateTime/DateTime64: In scope SELECT subtractMinutes([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT subtractMinutes([], [], []);) 2025.09.25 21:11:55.424672 [ 2707 ] {04af0601-6df2-4258-a9aa-02fd9a8aa8bb} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractMinutes. Must be a DateTime/DateTime64: In scope SELECT subtractMinutes([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:55.497503 [ 2707 ] {bc090dab-62b4-4ac3-8629-92a0d4cdc63c} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractMonths. Must be a DateTime/DateTime64: In scope SELECT subtractMonths([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT subtractMonths([], [], []);) 2025.09.25 21:11:55.509243 [ 2707 ] {bc090dab-62b4-4ac3-8629-92a0d4cdc63c} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractMonths. Must be a DateTime/DateTime64: In scope SELECT subtractMonths([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:55.548210 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:55.564468 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:55.778191 [ 2707 ] {ac42e4b5-78ff-4f61-9a93-d1a440ee2c46} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractNanoseconds. Must be a DateTime/DateTime64: In scope SELECT subtractNanoseconds([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT subtractNanoseconds([], [], []);) 2025.09.25 21:11:55.788877 [ 2707 ] {ac42e4b5-78ff-4f61-9a93-d1a440ee2c46} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractNanoseconds. Must be a DateTime/DateTime64: In scope SELECT subtractNanoseconds([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:55.882985 [ 2707 ] {cf0ae594-6908-438f-b299-0f05dc4be1ae} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractQuarters. Must be a DateTime/DateTime64: In scope SELECT subtractQuarters([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT subtractQuarters([], [], []);) 2025.09.25 21:11:55.889913 [ 2707 ] {cf0ae594-6908-438f-b299-0f05dc4be1ae} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractQuarters. Must be a DateTime/DateTime64: In scope SELECT subtractQuarters([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:56.015317 [ 2707 ] {5e1f035b-d4b8-4367-9843-5753bf770106} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractSeconds. Must be a DateTime/DateTime64: In scope SELECT subtractSeconds([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT subtractSeconds([], [], []);) 2025.09.25 21:11:56.025505 [ 2707 ] {5e1f035b-d4b8-4367-9843-5753bf770106} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractSeconds. Must be a DateTime/DateTime64: In scope SELECT subtractSeconds([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:56.122500 [ 2707 ] {a16a9591-8554-43dd-bfad-b42ac3e994a8} executeQuery: Code: 42. DB::Exception: Number of arguments for function subtractTupleOfIntervals doesn't match: passed 3, should be 2: In scope SELECT subtractTupleOfIntervals([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT subtractTupleOfIntervals([], [], []);) 2025.09.25 21:11:56.127121 [ 2707 ] {a16a9591-8554-43dd-bfad-b42ac3e994a8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function subtractTupleOfIntervals doesn't match: passed 3, should be 2: In scope SELECT subtractTupleOfIntervals([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:56.205361 [ 2707 ] {ef7f414b-8312-4513-a7a3-049c145b0195} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractWeeks. Must be a DateTime/DateTime64: In scope SELECT subtractWeeks([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT subtractWeeks([], [], []);) 2025.09.25 21:11:56.214282 [ 2707 ] {ef7f414b-8312-4513-a7a3-049c145b0195} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractWeeks. Must be a DateTime/DateTime64: In scope SELECT subtractWeeks([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:56.288395 [ 2707 ] {c5f40d61-5c0e-45de-a174-4776a8850ec2} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractYears. Must be a DateTime/DateTime64: In scope SELECT subtractYears([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT subtractYears([], [], []);) 2025.09.25 21:11:56.293994 [ 2707 ] {c5f40d61-5c0e-45de-a174-4776a8850ec2} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function subtractYears. Must be a DateTime/DateTime64: In scope SELECT subtractYears([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:56.334904 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:56.393256 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:56.400325 [ 2707 ] {50111462-f573-4f0e-8098-3cc83a7dee38} executeQuery: Code: 42. DB::Exception: Aggregate function sum requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sum([], [], []);) 2025.09.25 21:11:56.408397 [ 2707 ] {50111462-f573-4f0e-8098-3cc83a7dee38} TCPHandler: Code: 42. DB::Exception: Aggregate function sum requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:56.538484 [ 2707 ] {abd6eec4-2d1e-4375-9b49-c34a5384f212} executeQuery: Code: 42. DB::Exception: Aggregate function sumCount requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sumCount([], [], []);) 2025.09.25 21:11:56.550562 [ 2707 ] {abd6eec4-2d1e-4375-9b49-c34a5384f212} TCPHandler: Code: 42. DB::Exception: Aggregate function sumCount requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:56.654201 [ 2707 ] {87207260-143a-4c2f-a2c1-1fe033727040} executeQuery: Code: 42. DB::Exception: Aggregate function sumKahan requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sumKahan([], [], []);) 2025.09.25 21:11:56.661914 [ 2707 ] {87207260-143a-4c2f-a2c1-1fe033727040} TCPHandler: Code: 42. DB::Exception: Aggregate function sumKahan requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:56.734598 [ 2707 ] {9d607037-ed91-42f7-80ee-642a4701882b} executeQuery: Code: 43. DB::Exception: Values for -Map cannot be summed, passed type Nothing. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sumMapFiltered([], [], []);) 2025.09.25 21:11:56.741847 [ 2707 ] {9d607037-ed91-42f7-80ee-642a4701882b} TCPHandler: Code: 43. DB::Exception: Values for -Map cannot be summed, passed type Nothing. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:56.821003 [ 2707 ] {fb13cae6-d078-441c-af87-6c521f6b0c9c} executeQuery: Code: 43. DB::Exception: Values for -Map cannot be summed, passed type Nothing. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sumMapFilteredWithOverflow([], [], []);) 2025.09.25 21:11:56.827253 [ 2707 ] {fb13cae6-d078-441c-af87-6c521f6b0c9c} TCPHandler: Code: 43. DB::Exception: Values for -Map cannot be summed, passed type Nothing. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:56.926501 [ 2707 ] {15d82636-0b66-4cc9-bdd4-6c65cfe84610} executeQuery: Code: 43. DB::Exception: Values for -Map cannot be summed, passed type Nothing. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sumMapWithOverflow([], [], []);) 2025.09.25 21:11:56.938424 [ 2707 ] {15d82636-0b66-4cc9-bdd4-6c65cfe84610} TCPHandler: Code: 43. DB::Exception: Values for -Map cannot be summed, passed type Nothing. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:57.039719 [ 2707 ] {1d634774-9e1b-4c32-b7b0-483a282f7eeb} executeQuery: Code: 43. DB::Exception: Values for -Map cannot be summed, passed type Nothing. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sumMappedArrays([], [], []);) 2025.09.25 21:11:57.046892 [ 2707 ] {1d634774-9e1b-4c32-b7b0-483a282f7eeb} TCPHandler: Code: 43. DB::Exception: Values for -Map cannot be summed, passed type Nothing. (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:57.108635 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:57.160644 [ 2707 ] {8f4a6cac-f5ae-4d50-ad6e-6381515a0a48} executeQuery: Code: 42. DB::Exception: Aggregate function sumWithOverflow requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT sumWithOverflow([], [], []);) 2025.09.25 21:11:57.174533 [ 2707 ] {8f4a6cac-f5ae-4d50-ad6e-6381515a0a48} TCPHandler: Code: 42. DB::Exception: Aggregate function sumWithOverflow requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:57.184497 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:57.277274 [ 2707 ] {3c3f6e21-9412-47f1-8bac-939b3d15ffe3} executeQuery: Code: 42. DB::Exception: Incorrect number of arguments: expected 1 or 2 arguments: In scope SELECT svg([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT svg([], [], []);) 2025.09.25 21:11:57.290058 [ 2707 ] {3c3f6e21-9412-47f1-8bac-939b3d15ffe3} TCPHandler: Code: 42. DB::Exception: Incorrect number of arguments: expected 1 or 2 arguments: In scope SELECT svg([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:57.357678 [ 2707 ] {ef95d6a5-4c85-4b18-a132-92bb8d49d576} executeQuery: Code: 344. DB::Exception: Natural language processing function 'synonyms' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT synonyms([], [], []);) 2025.09.25 21:11:57.407936 [ 2707 ] {ef95d6a5-4c85-4b18-a132-92bb8d49d576} TCPHandler: Code: 344. DB::Exception: Natural language processing function 'synonyms' is experimental. Set `allow_experimental_nlp_functions` setting to enable it. (SUPPORT_IS_DISABLED) 2025.09.25 21:11:57.502616 [ 2707 ] {7513d0c6-ed40-40b2-bdd2-879d239cf26f} executeQuery: Code: 42. DB::Exception: Number of arguments for function tan doesn't match: passed 3, should be 1: In scope SELECT tan([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tan([], [], []);) 2025.09.25 21:11:57.515111 [ 2707 ] {7513d0c6-ed40-40b2-bdd2-879d239cf26f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tan doesn't match: passed 3, should be 1: In scope SELECT tan([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:57.625833 [ 2707 ] {1353bf3f-49ca-488b-a0d0-8c4319be540d} executeQuery: Code: 42. DB::Exception: Number of arguments for function tanh doesn't match: passed 3, should be 1: In scope SELECT tanh([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tanh([], [], []);) 2025.09.25 21:11:57.635277 [ 2707 ] {1353bf3f-49ca-488b-a0d0-8c4319be540d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tanh doesn't match: passed 3, should be 1: In scope SELECT tanh([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:57.722525 [ 2707 ] {97e0e64c-e00c-4f1c-b080-833077da4092} executeQuery: Code: 42. DB::Exception: Number of arguments for function tcpPort doesn't match: passed 3, should be 0: In scope SELECT tcpPort([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tcpPort([], [], []);) 2025.09.25 21:11:57.733930 [ 2707 ] {97e0e64c-e00c-4f1c-b080-833077da4092} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tcpPort doesn't match: passed 3, should be 0: In scope SELECT tcpPort([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:57.814990 [ 3619 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:57.827596 [ 2707 ] {e443fde8-c866-4c82-8a3f-b7380af99f02} executeQuery: Code: 42. DB::Exception: Number of arguments for function test_function doesn't match: passed 3, should be 2: In scope SELECT test_function([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT test_function([], [], []);) 2025.09.25 21:11:57.836006 [ 2707 ] {e443fde8-c866-4c82-8a3f-b7380af99f02} TCPHandler: Code: 42. DB::Exception: Number of arguments for function test_function doesn't match: passed 3, should be 2: In scope SELECT test_function([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:57.879679 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:57.901601 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:57.926249 [ 2707 ] {5b331f89-b9a9-4b21-b78c-ee275d80f0c4} executeQuery: Code: 42. DB::Exception: Number of arguments for function tgamma doesn't match: passed 3, should be 1: In scope SELECT tgamma([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tgamma([], [], []);) 2025.09.25 21:11:57.933418 [ 2707 ] {5b331f89-b9a9-4b21-b78c-ee275d80f0c4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tgamma doesn't match: passed 3, should be 1: In scope SELECT tgamma([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:58.021223 [ 2707 ] {73bb44f9-1f10-4d89-a985-8e9ed59d0ac5} executeQuery: Code: 42. DB::Exception: Aggregate function theilsU requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT theilsU([], [], []);) 2025.09.25 21:11:58.033321 [ 2707 ] {73bb44f9-1f10-4d89-a985-8e9ed59d0ac5} TCPHandler: Code: 42. DB::Exception: Aggregate function theilsU requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:58.156847 [ 2707 ] {c79f63f7-b0d9-4e9d-86bf-bb4ade45e7bd} executeQuery: Code: 42. DB::Exception: Number of arguments for function throwIf doesn't match: passed 3, should be 1 or 2: In scope SELECT throwIf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT throwIf([], [], []);) 2025.09.25 21:11:58.163301 [ 2707 ] {c79f63f7-b0d9-4e9d-86bf-bb4ade45e7bd} TCPHandler: Code: 42. DB::Exception: Number of arguments for function throwIf doesn't match: passed 3, should be 1 or 2: In scope SELECT throwIf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:58.257144 [ 2707 ] {13199dbd-98a8-4d75-84a7-13fb2e020cbe} executeQuery: Code: 42. DB::Exception: Number of arguments for function tid doesn't match: passed 3, should be 0: In scope SELECT tid([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tid([], [], []);) 2025.09.25 21:11:58.262718 [ 2707 ] {13199dbd-98a8-4d75-84a7-13fb2e020cbe} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tid doesn't match: passed 3, should be 0: In scope SELECT tid([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:58.350224 [ 2707 ] {fd72b4d8-dbf2-41cc-838e-d5fd9c13ccb6} executeQuery: Code: 42. DB::Exception: Number of arguments for function timeDiff doesn't match: passed 3, should be 2: In scope SELECT timeDiff([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT timeDiff([], [], []);) 2025.09.25 21:11:58.361201 [ 2707 ] {fd72b4d8-dbf2-41cc-838e-d5fd9c13ccb6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function timeDiff doesn't match: passed 3, should be 2: In scope SELECT timeDiff([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:58.462814 [ 2707 ] {1e8d05d8-73a6-4107-b38b-4600b140bc62} executeQuery: Code: 42. DB::Exception: Number of arguments for function timeSlot doesn't match: passed 3, should be 1 or 2: In scope SELECT timeSlot([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT timeSlot([], [], []);) 2025.09.25 21:11:58.475529 [ 2707 ] {1e8d05d8-73a6-4107-b38b-4600b140bc62} TCPHandler: Code: 42. DB::Exception: Number of arguments for function timeSlot doesn't match: passed 3, should be 1 or 2: In scope SELECT timeSlot([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:58.547618 [ 2707 ] {feca197d-3b86-4c62-a873-1136bd2a4ef0} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function timeSlots. Must be DateTime or DateTime64.: In scope SELECT timeSlots([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT timeSlots([], [], []);) 2025.09.25 21:11:58.564166 [ 2707 ] {feca197d-3b86-4c62-a873-1136bd2a4ef0} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function timeSlots. Must be DateTime or DateTime64.: In scope SELECT timeSlots([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:58.654091 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:58.673704 [ 2707 ] {436db578-9b32-4b3e-823f-861e7ace3121} executeQuery: Code: 42. DB::Exception: Number of arguments for function timezone doesn't match: passed 3, should be 0: In scope SELECT timeZone([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT timeZone([], [], []);) 2025.09.25 21:11:58.685322 [ 2707 ] {436db578-9b32-4b3e-823f-861e7ace3121} TCPHandler: Code: 42. DB::Exception: Number of arguments for function timezone doesn't match: passed 3, should be 0: In scope SELECT timeZone([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:58.752530 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:58.770554 [ 2707 ] {d2e973d6-473d-4c91-b9ad-01d5c8d1f887} executeQuery: Code: 42. DB::Exception: Number of arguments for function timezoneOf doesn't match: passed 3, should be 1: In scope SELECT timeZoneOf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT timeZoneOf([], [], []);) 2025.09.25 21:11:58.774888 [ 2707 ] {d2e973d6-473d-4c91-b9ad-01d5c8d1f887} TCPHandler: Code: 42. DB::Exception: Number of arguments for function timezoneOf doesn't match: passed 3, should be 1: In scope SELECT timeZoneOf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:58.910175 [ 2707 ] {372bf90e-cd0f-47a7-9fec-c98a4d4ef279} executeQuery: Code: 42. DB::Exception: Number of arguments for function timezoneOffset doesn't match: passed 3, should be 1 or 2: In scope SELECT timeZoneOffset([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT timeZoneOffset([], [], []);) 2025.09.25 21:11:58.922430 [ 2707 ] {372bf90e-cd0f-47a7-9fec-c98a4d4ef279} TCPHandler: Code: 42. DB::Exception: Number of arguments for function timezoneOffset doesn't match: passed 3, should be 1 or 2: In scope SELECT timeZoneOffset([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:58.988096 [ 2707 ] {07bed764-7682-46e0-a4a6-024594410dc8} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'timestamp'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT timestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT timestamp([], [], []);) 2025.09.25 21:11:58.995393 [ 2707 ] {07bed764-7682-46e0-a4a6-024594410dc8} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'timestamp'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT timestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:59.078470 [ 2707 ] {8740365e-7373-453e-bb21-e0ebc3d5d518} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'unit' to function 'dateDiff'. Expected: String, got: Array(Nothing): In scope SELECT dateDiff([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT timestampDiff([], [], []);) 2025.09.25 21:11:59.085798 [ 2707 ] {8740365e-7373-453e-bb21-e0ebc3d5d518} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'unit' to function 'dateDiff'. Expected: String, got: Array(Nothing): In scope SELECT dateDiff([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:59.140041 [ 2707 ] {2258570a-82d8-4f60-9d74-1a70ea37d447} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'unit' to function 'dateDiff'. Expected: String, got: Array(Nothing): In scope SELECT dateDiff([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT timestamp_diff([], [], []);) 2025.09.25 21:11:59.152101 [ 2707 ] {2258570a-82d8-4f60-9d74-1a70ea37d447} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'unit' to function 'dateDiff'. Expected: String, got: Array(Nothing): In scope SELECT dateDiff([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:11:59.152926 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:59.203016 [ 2707 ] {54cf8761-b5bd-4bb0-8978-b33a3b6d6182} executeQuery: Code: 42. DB::Exception: Number of arguments for function timezone doesn't match: passed 3, should be 0: In scope SELECT timezone([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT timezone([], [], []);) 2025.09.25 21:11:59.207754 [ 2707 ] {54cf8761-b5bd-4bb0-8978-b33a3b6d6182} TCPHandler: Code: 42. DB::Exception: Number of arguments for function timezone doesn't match: passed 3, should be 0: In scope SELECT timezone([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:59.217081 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:59.302002 [ 2707 ] {b7645c80-06f1-4609-a3c0-876fde3cc9b2} executeQuery: Code: 42. DB::Exception: Number of arguments for function timezoneOf doesn't match: passed 3, should be 1: In scope SELECT timezoneOf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT timezoneOf([], [], []);) 2025.09.25 21:11:59.321961 [ 2707 ] {b7645c80-06f1-4609-a3c0-876fde3cc9b2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function timezoneOf doesn't match: passed 3, should be 1: In scope SELECT timezoneOf([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:59.392201 [ 2707 ] {81c78d47-8a9b-4e48-be2a-835faddd8788} executeQuery: Code: 42. DB::Exception: Number of arguments for function timezoneOffset doesn't match: passed 3, should be 1 or 2: In scope SELECT timezoneOffset([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT timezoneOffset([], [], []);) 2025.09.25 21:11:59.404165 [ 2707 ] {81c78d47-8a9b-4e48-be2a-835faddd8788} TCPHandler: Code: 42. DB::Exception: Number of arguments for function timezoneOffset doesn't match: passed 3, should be 1 or 2: In scope SELECT timezoneOffset([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:59.479805 [ 2707 ] {f3cf9cc6-6d09-42c9-ad60-9f9f405f422e} executeQuery: Code: 42. DB::Exception: Number of arguments for function toBool doesn't match: passed 3, should be 1: In scope SELECT toBool([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toBool([], [], []);) 2025.09.25 21:11:59.497470 [ 2707 ] {f3cf9cc6-6d09-42c9-ad60-9f9f405f422e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toBool doesn't match: passed 3, should be 1: In scope SELECT toBool([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:59.577872 [ 2707 ] {e54a85ff-af0b-412e-b8fc-8c974a7a6e48} executeQuery: Code: 42. DB::Exception: Number of arguments for function toColumnTypeName doesn't match: passed 3, should be 1: In scope SELECT toColumnTypeName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toColumnTypeName([], [], []);) 2025.09.25 21:11:59.583535 [ 2707 ] {e54a85ff-af0b-412e-b8fc-8c974a7a6e48} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toColumnTypeName doesn't match: passed 3, should be 1: In scope SELECT toColumnTypeName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:59.642278 [ 2707 ] {e2f27eda-5b76-477c-847b-9f7b82778adf} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDate'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toDate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDate([], [], []);) 2025.09.25 21:11:59.652622 [ 2707 ] {e2f27eda-5b76-477c-847b-9f7b82778adf} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDate'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toDate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:59.730900 [ 2707 ] {54fd8c3e-39a1-48c6-9104-8b7327f4dcaf} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDate32'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toDate32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDate32([], [], []);) 2025.09.25 21:11:59.741277 [ 2707 ] {54fd8c3e-39a1-48c6-9104-8b7327f4dcaf} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDate32'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toDate32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:59.782332 [ 3356 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {8b990b7c-a181-415f-9d90-25ec9d0f3599::202509_1_66_13}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:11:59.809266 [ 2707 ] {d71b0d85-a764-46b5-9113-1110ae0b8efa} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDate32OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toDate32OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDate32OrDefault([], [], []);) 2025.09.25 21:11:59.819222 [ 2707 ] {d71b0d85-a764-46b5-9113-1110ae0b8efa} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDate32OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toDate32OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:11:59.954159 [ 2707 ] {e129a6c5-cd3d-4e1c-b11b-5ba378cfe93c} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDate32OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDate32OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDate32OrNull([], [], []);) 2025.09.25 21:11:59.957074 [ 2707 ] {e129a6c5-cd3d-4e1c-b11b-5ba378cfe93c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDate32OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDate32OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:00.018979 [ 2707 ] {70665dfa-abac-4372-938c-82d063cad7a3} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDate32OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDate32OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDate32OrZero([], [], []);) 2025.09.25 21:12:00.023743 [ 2707 ] {70665dfa-abac-4372-938c-82d063cad7a3} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDate32OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDate32OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:00.026552 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:00.052861 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:00.086329 [ 2707 ] {bf47c563-c438-41dd-a962-367297e41fcb} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDateOrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toDateOrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDateOrDefault([], [], []);) 2025.09.25 21:12:00.090365 [ 2707 ] {bf47c563-c438-41dd-a962-367297e41fcb} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDateOrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toDateOrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:00.174672 [ 2707 ] {a3626f31-4fe5-406e-aec5-a2da99fca3d6} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDateOrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDateOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDateOrNull([], [], []);) 2025.09.25 21:12:00.185116 [ 2707 ] {a3626f31-4fe5-406e-aec5-a2da99fca3d6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDateOrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDateOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:00.333234 [ 2707 ] {3e0be033-796f-4057-ae38-a213dd0a3d7d} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDateOrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDateOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDateOrZero([], [], []);) 2025.09.25 21:12:00.339019 [ 2707 ] {3e0be033-796f-4057-ae38-a213dd0a3d7d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDateOrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDateOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:00.439148 [ 2707 ] {f16609fd-2082-4739-a282-86deb3ae2343} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'scale' to function 'toDateTime'. Expected: const Integer, got: Array(Nothing): In scope SELECT toDateTime([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDateTime([], [], []);) 2025.09.25 21:12:00.444950 [ 2707 ] {f16609fd-2082-4739-a282-86deb3ae2343} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'scale' to function 'toDateTime'. Expected: const Integer, got: Array(Nothing): In scope SELECT toDateTime([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:00.524163 [ 2707 ] {fb79337a-4e66-4288-a5c1-a990a25a752b} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDateTime32'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toDateTime32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDateTime32([], [], []);) 2025.09.25 21:12:00.529087 [ 2707 ] {fb79337a-4e66-4288-a5c1-a990a25a752b} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDateTime32'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toDateTime32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:00.725974 [ 2707 ] {45cab725-e55e-43bf-a11d-50a78ae88f49} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'scale' to function 'toDateTime64'. Expected: const Integer, got: Array(Nothing): In scope SELECT toDateTime64([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDateTime64([], [], []);) 2025.09.25 21:12:00.738274 [ 2707 ] {45cab725-e55e-43bf-a11d-50a78ae88f49} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'scale' to function 'toDateTime64'. Expected: const Integer, got: Array(Nothing): In scope SELECT toDateTime64([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:00.749086 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:00.779148 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:00.822395 [ 2707 ] {0a6ebb5b-3ba6-49dd-b31c-a549c936be26} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'scale' to function 'toDateTime64OrDefault'. Expected: const Integer, got: Array(Nothing): In scope SELECT toDateTime64OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDateTime64OrDefault([], [], []);) 2025.09.25 21:12:00.830133 [ 2707 ] {0a6ebb5b-3ba6-49dd-b31c-a549c936be26} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'scale' to function 'toDateTime64OrDefault'. Expected: const Integer, got: Array(Nothing): In scope SELECT toDateTime64OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:00.925123 [ 2707 ] {21fd80da-27bb-44f7-8c48-f4959d0df007} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'toDateTime64OrNull'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT toDateTime64OrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDateTime64OrNull([], [], []);) 2025.09.25 21:12:00.937405 [ 2707 ] {21fd80da-27bb-44f7-8c48-f4959d0df007} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'toDateTime64OrNull'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT toDateTime64OrNull([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:01.019692 [ 2707 ] {69c83287-9688-4a50-b1d0-d57688408a0a} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'toDateTime64OrZero'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT toDateTime64OrZero([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDateTime64OrZero([], [], []);) 2025.09.25 21:12:01.038337 [ 2707 ] {69c83287-9688-4a50-b1d0-d57688408a0a} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'string' to function 'toDateTime64OrZero'. Expected: String or FixedString, got: Array(Nothing): In scope SELECT toDateTime64OrZero([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:01.168838 [ 2707 ] {847abba4-7191-49e6-be7d-359a12aa0151} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'timezone' to function 'toDateTimeOrDefault'. Expected: const String, got: Array(Nothing): In scope SELECT toDateTimeOrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDateTimeOrDefault([], [], []);) 2025.09.25 21:12:01.170867 [ 2707 ] {847abba4-7191-49e6-be7d-359a12aa0151} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'timezone' to function 'toDateTimeOrDefault'. Expected: const String, got: Array(Nothing): In scope SELECT toDateTimeOrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:01.246874 [ 2707 ] {7a34e575-c643-4874-9893-ef457840b5e8} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDateTimeOrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDateTimeOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDateTimeOrNull([], [], []);) 2025.09.25 21:12:01.256734 [ 2707 ] {7a34e575-c643-4874-9893-ef457840b5e8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDateTimeOrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDateTimeOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:01.422212 [ 2707 ] {2bd75020-aa18-4028-be56-bab3d0886441} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDateTimeOrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDateTimeOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDateTimeOrZero([], [], []);) 2025.09.25 21:12:01.431189 [ 2707 ] {2bd75020-aa18-4028-be56-bab3d0886441} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDateTimeOrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDateTimeOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:01.536038 [ 2707 ] {387a5b84-52d9-4a3c-a18d-7574c8160cf1} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDayOfMonth doesn't match: passed 3, should be 1 or 2: In scope SELECT toDayOfMonth([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDayOfMonth([], [], []);) 2025.09.25 21:12:01.543832 [ 2707 ] {387a5b84-52d9-4a3c-a18d-7574c8160cf1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDayOfMonth doesn't match: passed 3, should be 1 or 2: In scope SELECT toDayOfMonth([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:01.613131 [ 2707 ] {5dd601a7-0ef0-4519-97d5-62b90dc64485} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function toDayOfWeek. Must be Date, Date32, DateTime or DateTime64: In scope SELECT toDayOfWeek([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDayOfWeek([], [], []);) 2025.09.25 21:12:01.618536 [ 2707 ] {5dd601a7-0ef0-4519-97d5-62b90dc64485} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function toDayOfWeek. Must be Date, Date32, DateTime or DateTime64: In scope SELECT toDayOfWeek([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:01.685162 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:01.713326 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:01.759043 [ 2707 ] {c38ced37-dc86-4194-83fe-786d531c52bb} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDayOfYear doesn't match: passed 3, should be 1 or 2: In scope SELECT toDayOfYear([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDayOfYear([], [], []);) 2025.09.25 21:12:01.761416 [ 2707 ] {c38ced37-dc86-4194-83fe-786d531c52bb} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDayOfYear doesn't match: passed 3, should be 1 or 2: In scope SELECT toDayOfYear([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:01.813764 [ 2707 ] {cb6e2465-f398-4a72-81fe-42d046668d4f} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDaysSinceYearZero doesn't match: passed 3, should be 1 or 2: In scope SELECT toDaysSinceYearZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDaysSinceYearZero([], [], []);) 2025.09.25 21:12:01.844254 [ 2707 ] {cb6e2465-f398-4a72-81fe-42d046668d4f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDaysSinceYearZero doesn't match: passed 3, should be 1 or 2: In scope SELECT toDaysSinceYearZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:01.940512 [ 2707 ] {1d93295f-8c67-4e4d-b05e-b8e91dc99d9d} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDecimal128'. Expected 2 arguments, got 3 arguments: In scope SELECT toDecimal128([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimal128([], [], []);) 2025.09.25 21:12:01.948316 [ 2707 ] {1d93295f-8c67-4e4d-b05e-b8e91dc99d9d} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDecimal128'. Expected 2 arguments, got 3 arguments: In scope SELECT toDecimal128([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:02.014654 [ 2707 ] {2fb53b8f-f24c-4980-9028-220b3542f7fb} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'scale' to function 'toDecimal128OrDefault'. Expected: const Integer, got: Array(Nothing): In scope SELECT toDecimal128OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimal128OrDefault([], [], []);) 2025.09.25 21:12:02.019654 [ 2707 ] {2fb53b8f-f24c-4980-9028-220b3542f7fb} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'scale' to function 'toDecimal128OrDefault'. Expected: const Integer, got: Array(Nothing): In scope SELECT toDecimal128OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:02.105245 [ 2707 ] {fafe65b3-88c3-4ff0-b9dd-81f585cc6551} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDecimal128OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDecimal128OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimal128OrNull([], [], []);) 2025.09.25 21:12:02.111763 [ 2707 ] {fafe65b3-88c3-4ff0-b9dd-81f585cc6551} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDecimal128OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDecimal128OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:02.205130 [ 2707 ] {90b2e475-a257-400c-8cce-38973772bb3f} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDecimal128OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDecimal128OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimal128OrZero([], [], []);) 2025.09.25 21:12:02.207631 [ 2707 ] {90b2e475-a257-400c-8cce-38973772bb3f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDecimal128OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDecimal128OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:02.265670 [ 2707 ] {ae8329ff-2105-4ee8-a0ad-aee5b0fa7a99} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDecimal256'. Expected 2 arguments, got 3 arguments: In scope SELECT toDecimal256([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimal256([], [], []);) 2025.09.25 21:12:02.271398 [ 2707 ] {ae8329ff-2105-4ee8-a0ad-aee5b0fa7a99} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDecimal256'. Expected 2 arguments, got 3 arguments: In scope SELECT toDecimal256([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:02.331199 [ 2707 ] {64cde3cb-3563-4521-941e-a0f91952e6f3} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'scale' to function 'toDecimal256OrDefault'. Expected: const Integer, got: Array(Nothing): In scope SELECT toDecimal256OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimal256OrDefault([], [], []);) 2025.09.25 21:12:02.339389 [ 2707 ] {64cde3cb-3563-4521-941e-a0f91952e6f3} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'scale' to function 'toDecimal256OrDefault'. Expected: const Integer, got: Array(Nothing): In scope SELECT toDecimal256OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:02.532960 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:02.560011 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:02.612604 [ 2707 ] {80dfe4c9-54eb-4208-b7c0-e45d2d0bcadc} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDecimal256OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDecimal256OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimal256OrNull([], [], []);) 2025.09.25 21:12:02.632702 [ 2707 ] {80dfe4c9-54eb-4208-b7c0-e45d2d0bcadc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDecimal256OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDecimal256OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:02.776586 [ 2707 ] {c92400d1-e42e-4a9f-a448-6e6e771c3878} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDecimal256OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDecimal256OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimal256OrZero([], [], []);) 2025.09.25 21:12:02.799192 [ 2707 ] {c92400d1-e42e-4a9f-a448-6e6e771c3878} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDecimal256OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDecimal256OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:02.891624 [ 2707 ] {58b82ff2-6c33-463c-a004-b113e47c3089} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDecimal32'. Expected 2 arguments, got 3 arguments: In scope SELECT toDecimal32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimal32([], [], []);) 2025.09.25 21:12:02.898194 [ 2707 ] {58b82ff2-6c33-463c-a004-b113e47c3089} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDecimal32'. Expected 2 arguments, got 3 arguments: In scope SELECT toDecimal32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:03.046157 [ 2707 ] {203ac855-9afa-466b-be9e-a1cd02242191} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'scale' to function 'toDecimal32OrDefault'. Expected: const Integer, got: Array(Nothing): In scope SELECT toDecimal32OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimal32OrDefault([], [], []);) 2025.09.25 21:12:03.056691 [ 2707 ] {203ac855-9afa-466b-be9e-a1cd02242191} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'scale' to function 'toDecimal32OrDefault'. Expected: const Integer, got: Array(Nothing): In scope SELECT toDecimal32OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:03.205208 [ 2707 ] {aa345c78-f041-43dc-b2df-d4d332e800ad} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDecimal32OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDecimal32OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimal32OrNull([], [], []);) 2025.09.25 21:12:03.211844 [ 2707 ] {aa345c78-f041-43dc-b2df-d4d332e800ad} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDecimal32OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDecimal32OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:03.422741 [ 2707 ] {6ae4d8b3-6e3a-406f-8a5f-9a889e06927c} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDecimal32OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDecimal32OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimal32OrZero([], [], []);) 2025.09.25 21:12:03.430483 [ 2707 ] {6ae4d8b3-6e3a-406f-8a5f-9a889e06927c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDecimal32OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDecimal32OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:03.503179 [ 2707 ] {db6e399f-c50c-49db-bb6a-002f2e37aa00} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDecimal64'. Expected 2 arguments, got 3 arguments: In scope SELECT toDecimal64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimal64([], [], []);) 2025.09.25 21:12:03.511587 [ 2707 ] {db6e399f-c50c-49db-bb6a-002f2e37aa00} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toDecimal64'. Expected 2 arguments, got 3 arguments: In scope SELECT toDecimal64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:03.581672 [ 2707 ] {caba429f-7859-40af-994b-30862aca6198} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'scale' to function 'toDecimal64OrDefault'. Expected: const Integer, got: Array(Nothing): In scope SELECT toDecimal64OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimal64OrDefault([], [], []);) 2025.09.25 21:12:03.591805 [ 2707 ] {caba429f-7859-40af-994b-30862aca6198} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'scale' to function 'toDecimal64OrDefault'. Expected: const Integer, got: Array(Nothing): In scope SELECT toDecimal64OrDefault([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:03.657727 [ 2707 ] {b71ba5dd-01f7-40f2-98e8-4b8c0c5ba7e8} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDecimal64OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDecimal64OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimal64OrNull([], [], []);) 2025.09.25 21:12:03.666058 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:03.666275 [ 2707 ] {b71ba5dd-01f7-40f2-98e8-4b8c0c5ba7e8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDecimal64OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDecimal64OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:03.690480 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:03.743811 [ 2707 ] {c1e9b1df-478a-4768-af1c-3ee5250e6b64} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDecimal64OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDecimal64OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimal64OrZero([], [], []);) 2025.09.25 21:12:03.754791 [ 2707 ] {c1e9b1df-478a-4768-af1c-3ee5250e6b64} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDecimal64OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toDecimal64OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:03.856678 [ 2707 ] {83c16eb6-1099-4002-9781-f1382f822841} executeQuery: Code: 42. DB::Exception: Number of arguments for function toDecimalString doesn't match: passed 3, should be 2: In scope SELECT toDecimalString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toDecimalString([], [], []);) 2025.09.25 21:12:03.861410 [ 2707 ] {83c16eb6-1099-4002-9781-f1382f822841} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toDecimalString doesn't match: passed 3, should be 2: In scope SELECT toDecimalString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:03.950003 [ 2707 ] {71b86021-b122-4ca3-9814-3ca7aec481b0} executeQuery: Code: 42. DB::Exception: Number of arguments for function toFixedString doesn't match: passed 3, should be 2: In scope SELECT toFixedString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toFixedString([], [], []);) 2025.09.25 21:12:03.966486 [ 2707 ] {71b86021-b122-4ca3-9814-3ca7aec481b0} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toFixedString doesn't match: passed 3, should be 2: In scope SELECT toFixedString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:04.044665 [ 2707 ] {f4389d4f-7d51-44a8-9c11-065f86bc7821} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toFloat32'. Expected 1 argument, got 3 arguments: In scope SELECT toFloat32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toFloat32([], [], []);) 2025.09.25 21:12:04.052559 [ 2707 ] {f4389d4f-7d51-44a8-9c11-065f86bc7821} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toFloat32'. Expected 1 argument, got 3 arguments: In scope SELECT toFloat32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:04.134631 [ 2707 ] {5e896295-8de5-41a4-88d0-a0377ad544df} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toFloat32OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toFloat32OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toFloat32OrDefault([], [], []);) 2025.09.25 21:12:04.140940 [ 2707 ] {5e896295-8de5-41a4-88d0-a0377ad544df} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toFloat32OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toFloat32OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:04.220972 [ 2707 ] {383d36b4-9e95-4660-883f-3f78d86e5747} executeQuery: Code: 42. DB::Exception: Number of arguments for function toFloat32OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toFloat32OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toFloat32OrNull([], [], []);) 2025.09.25 21:12:04.227194 [ 2707 ] {383d36b4-9e95-4660-883f-3f78d86e5747} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toFloat32OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toFloat32OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:04.371772 [ 2707 ] {7e2f60f0-d648-4ec4-8897-d206378ab2dc} executeQuery: Code: 42. DB::Exception: Number of arguments for function toFloat32OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toFloat32OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toFloat32OrZero([], [], []);) 2025.09.25 21:12:04.377627 [ 2707 ] {7e2f60f0-d648-4ec4-8897-d206378ab2dc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toFloat32OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toFloat32OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:04.443123 [ 2707 ] {c3b0f3fb-f7b7-44dc-829a-67c450ca79c4} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toFloat64'. Expected 1 argument, got 3 arguments: In scope SELECT toFloat64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toFloat64([], [], []);) 2025.09.25 21:12:04.453958 [ 2707 ] {c3b0f3fb-f7b7-44dc-829a-67c450ca79c4} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toFloat64'. Expected 1 argument, got 3 arguments: In scope SELECT toFloat64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:04.514544 [ 2707 ] {a11507c1-3eb2-4e6b-8813-35544b271314} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toFloat64OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toFloat64OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toFloat64OrDefault([], [], []);) 2025.09.25 21:12:04.518920 [ 2707 ] {a11507c1-3eb2-4e6b-8813-35544b271314} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toFloat64OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toFloat64OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:04.587682 [ 2707 ] {36c1f999-3235-4573-a6d1-ca14c57fc4d6} executeQuery: Code: 42. DB::Exception: Number of arguments for function toFloat64OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toFloat64OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toFloat64OrNull([], [], []);) 2025.09.25 21:12:04.592502 [ 2707 ] {36c1f999-3235-4573-a6d1-ca14c57fc4d6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toFloat64OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toFloat64OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:04.666573 [ 2707 ] {5d435990-dfe5-4cb0-a621-ce56b507d872} executeQuery: Code: 42. DB::Exception: Number of arguments for function toFloat64OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toFloat64OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toFloat64OrZero([], [], []);) 2025.09.25 21:12:04.669880 [ 2707 ] {5d435990-dfe5-4cb0-a621-ce56b507d872} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toFloat64OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toFloat64OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:04.684419 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:04.712445 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:04.780939 [ 2707 ] {a98804bc-8240-467e-973a-820773b06ab0} executeQuery: Code: 42. DB::Exception: Number of arguments for function toHour doesn't match: passed 3, should be 1 or 2: In scope SELECT toHour([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toHour([], [], []);) 2025.09.25 21:12:04.784900 [ 2707 ] {a98804bc-8240-467e-973a-820773b06ab0} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toHour doesn't match: passed 3, should be 1 or 2: In scope SELECT toHour([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:04.880619 [ 2707 ] {4cd8f505-ff91-4eb0-a2d2-4e7d92527ead} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIPv4'. Expected 1 argument, got 3 arguments: In scope SELECT toIPv4([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIPv4([], [], []);) 2025.09.25 21:12:04.892971 [ 2707 ] {4cd8f505-ff91-4eb0-a2d2-4e7d92527ead} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIPv4'. Expected 1 argument, got 3 arguments: In scope SELECT toIPv4([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:04.995738 [ 2707 ] {3eae1ff2-7107-4b59-bc34-2bd333d62951} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIPv4OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toIPv4OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIPv4OrDefault([], [], []);) 2025.09.25 21:12:05.010344 [ 2707 ] {3eae1ff2-7107-4b59-bc34-2bd333d62951} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIPv4OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toIPv4OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:05.172709 [ 2707 ] {40c4a45d-3547-463c-9fd0-b10e8eceef49} executeQuery: Code: 42. DB::Exception: Number of arguments for function toIPv4OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toIPv4OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIPv4OrNull([], [], []);) 2025.09.25 21:12:05.191666 [ 2707 ] {40c4a45d-3547-463c-9fd0-b10e8eceef49} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toIPv4OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toIPv4OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:05.232291 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:05.260177 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:05.276133 [ 2707 ] {94d779b0-83e7-4df6-8a15-99861611244b} executeQuery: Code: 42. DB::Exception: Number of arguments for function toIPv4OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toIPv4OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIPv4OrZero([], [], []);) 2025.09.25 21:12:05.283489 [ 2707 ] {94d779b0-83e7-4df6-8a15-99861611244b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toIPv4OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toIPv4OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:05.379607 [ 2707 ] {ddfd2379-ad53-4d46-b3c0-a95676c9594a} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIPv6'. Expected 1 argument, got 3 arguments: In scope SELECT toIPv6([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIPv6([], [], []);) 2025.09.25 21:12:05.393945 [ 2707 ] {ddfd2379-ad53-4d46-b3c0-a95676c9594a} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIPv6'. Expected 1 argument, got 3 arguments: In scope SELECT toIPv6([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:05.444950 [ 2707 ] {f747382c-17ba-4908-8ce6-86146f502d25} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIPv6OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toIPv6OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIPv6OrDefault([], [], []);) 2025.09.25 21:12:05.450170 [ 2707 ] {f747382c-17ba-4908-8ce6-86146f502d25} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIPv6OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toIPv6OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:05.601381 [ 2707 ] {59fe0955-7afb-41de-9d50-ddd758ce9d1d} executeQuery: Code: 42. DB::Exception: Number of arguments for function toIPv6OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toIPv6OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIPv6OrNull([], [], []);) 2025.09.25 21:12:05.607815 [ 2707 ] {59fe0955-7afb-41de-9d50-ddd758ce9d1d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toIPv6OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toIPv6OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:05.677635 [ 2707 ] {5fbbe628-365d-4a20-9760-4d36d6ede375} executeQuery: Code: 42. DB::Exception: Number of arguments for function toIPv6OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toIPv6OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIPv6OrZero([], [], []);) 2025.09.25 21:12:05.681503 [ 2707 ] {5fbbe628-365d-4a20-9760-4d36d6ede375} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toIPv6OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toIPv6OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:05.757417 [ 2707 ] {7316b74e-92c4-442e-952c-7eb6dc92ad57} executeQuery: Code: 42. DB::Exception: Number of arguments for function toISOWeek doesn't match: passed 3, should be 1 or 2: In scope SELECT toISOWeek([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toISOWeek([], [], []);) 2025.09.25 21:12:05.766499 [ 2707 ] {7316b74e-92c4-442e-952c-7eb6dc92ad57} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toISOWeek doesn't match: passed 3, should be 1 or 2: In scope SELECT toISOWeek([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:05.819486 [ 2707 ] {c8ee0182-20fc-4dd7-98cf-1f86f63a66c4} executeQuery: Code: 42. DB::Exception: Number of arguments for function toISOYear doesn't match: passed 3, should be 1 or 2: In scope SELECT toISOYear([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toISOYear([], [], []);) 2025.09.25 21:12:05.830949 [ 2707 ] {c8ee0182-20fc-4dd7-98cf-1f86f63a66c4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toISOYear doesn't match: passed 3, should be 1 or 2: In scope SELECT toISOYear([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:05.904279 [ 2707 ] {4ba308e1-423f-4ed4-a047-9517881e8cc7} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt128'. Expected 1 argument, got 3 arguments: In scope SELECT toInt128([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt128([], [], []);) 2025.09.25 21:12:05.912223 [ 2707 ] {4ba308e1-423f-4ed4-a047-9517881e8cc7} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt128'. Expected 1 argument, got 3 arguments: In scope SELECT toInt128([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:05.962327 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:05.993553 [ 2707 ] {15937748-86f8-4701-a54b-e61e76b7589f} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt128OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toInt128OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt128OrDefault([], [], []);) 2025.09.25 21:12:06.009033 [ 2707 ] {15937748-86f8-4701-a54b-e61e76b7589f} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt128OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toInt128OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:06.027418 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:06.159072 [ 2707 ] {0f6a4146-dec2-4a49-b694-7a96faa367df} executeQuery: Code: 42. DB::Exception: Number of arguments for function toInt128OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt128OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt128OrNull([], [], []);) 2025.09.25 21:12:06.170321 [ 2707 ] {0f6a4146-dec2-4a49-b694-7a96faa367df} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toInt128OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt128OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:06.253372 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:06.264595 [ 2707 ] {968e1fd3-1575-4a66-a010-878e99617c5f} executeQuery: Code: 42. DB::Exception: Number of arguments for function toInt128OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt128OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt128OrZero([], [], []);) 2025.09.25 21:12:06.277905 [ 2707 ] {968e1fd3-1575-4a66-a010-878e99617c5f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toInt128OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt128OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:06.286577 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:06.364889 [ 2707 ] {51359006-8dc7-465f-839c-6f1db0c854cb} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt16'. Expected 1 argument, got 3 arguments: In scope SELECT toInt16([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt16([], [], []);) 2025.09.25 21:12:06.385556 [ 2707 ] {51359006-8dc7-465f-839c-6f1db0c854cb} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt16'. Expected 1 argument, got 3 arguments: In scope SELECT toInt16([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:06.590815 [ 2707 ] {46d6a3ca-f2e4-4b64-aef9-1ff6baa3b29e} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt16OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toInt16OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt16OrDefault([], [], []);) 2025.09.25 21:12:06.596893 [ 2707 ] {46d6a3ca-f2e4-4b64-aef9-1ff6baa3b29e} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt16OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toInt16OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:06.776662 [ 2707 ] {0b8355be-e28d-4f57-8b3f-06b418968d32} executeQuery: Code: 42. DB::Exception: Number of arguments for function toInt16OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt16OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt16OrNull([], [], []);) 2025.09.25 21:12:06.786365 [ 2707 ] {0b8355be-e28d-4f57-8b3f-06b418968d32} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toInt16OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt16OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:06.890811 [ 2707 ] {82d300b1-e4be-4ac8-895f-05fb2204ce89} executeQuery: Code: 42. DB::Exception: Number of arguments for function toInt16OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt16OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt16OrZero([], [], []);) 2025.09.25 21:12:06.898919 [ 2707 ] {82d300b1-e4be-4ac8-895f-05fb2204ce89} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toInt16OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt16OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:07.017048 [ 2707 ] {74555591-5cb9-4b47-a85f-00cc5212a090} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt256'. Expected 1 argument, got 3 arguments: In scope SELECT toInt256([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt256([], [], []);) 2025.09.25 21:12:07.028175 [ 2707 ] {74555591-5cb9-4b47-a85f-00cc5212a090} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt256'. Expected 1 argument, got 3 arguments: In scope SELECT toInt256([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:07.123544 [ 2707 ] {20e4d7df-73e5-42ed-aaa8-9ad3bd129cd0} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt256OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toInt256OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt256OrDefault([], [], []);) 2025.09.25 21:12:07.131194 [ 2707 ] {20e4d7df-73e5-42ed-aaa8-9ad3bd129cd0} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt256OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toInt256OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:07.323973 [ 2707 ] {939b7415-0231-4108-aa2e-fbe8736af4c9} executeQuery: Code: 42. DB::Exception: Number of arguments for function toInt256OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt256OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt256OrNull([], [], []);) 2025.09.25 21:12:07.331961 [ 2707 ] {939b7415-0231-4108-aa2e-fbe8736af4c9} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toInt256OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt256OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:07.386317 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:07.432860 [ 2707 ] {28d7f2df-2602-46e9-b8c2-dd12cf6076f6} executeQuery: Code: 42. DB::Exception: Number of arguments for function toInt256OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt256OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt256OrZero([], [], []);) 2025.09.25 21:12:07.435329 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:07.440534 [ 2707 ] {28d7f2df-2602-46e9-b8c2-dd12cf6076f6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toInt256OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt256OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:07.515915 [ 2707 ] {e3f34c26-9736-418e-a77b-52158c404bb0} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt32'. Expected 1 argument, got 3 arguments: In scope SELECT toInt32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt32([], [], []);) 2025.09.25 21:12:07.526740 [ 2707 ] {e3f34c26-9736-418e-a77b-52158c404bb0} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt32'. Expected 1 argument, got 3 arguments: In scope SELECT toInt32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:07.634095 [ 2707 ] {bd740497-5f3d-44be-90b0-74193d151453} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt32OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toInt32OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt32OrDefault([], [], []);) 2025.09.25 21:12:07.649803 [ 2707 ] {bd740497-5f3d-44be-90b0-74193d151453} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt32OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toInt32OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:07.738170 [ 2707 ] {ff5744bf-0a0d-4ff8-b0bb-32e506b6f6f0} executeQuery: Code: 42. DB::Exception: Number of arguments for function toInt32OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt32OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt32OrNull([], [], []);) 2025.09.25 21:12:07.828421 [ 2707 ] {ff5744bf-0a0d-4ff8-b0bb-32e506b6f6f0} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toInt32OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt32OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:07.910061 [ 2707 ] {447278af-6393-4c4b-a6c4-0b42a09812e1} executeQuery: Code: 42. DB::Exception: Number of arguments for function toInt32OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt32OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt32OrZero([], [], []);) 2025.09.25 21:12:07.918622 [ 2707 ] {447278af-6393-4c4b-a6c4-0b42a09812e1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toInt32OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt32OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:08.025947 [ 2707 ] {6de55dcf-4d18-4b91-9dd6-d872ea3b9fe1} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt64'. Expected 1 argument, got 3 arguments: In scope SELECT toInt64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt64([], [], []);) 2025.09.25 21:12:08.033210 [ 2707 ] {6de55dcf-4d18-4b91-9dd6-d872ea3b9fe1} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt64'. Expected 1 argument, got 3 arguments: In scope SELECT toInt64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:08.143633 [ 2707 ] {c7500beb-ba9e-4922-91c3-de5f06fd2929} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt64OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toInt64OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt64OrDefault([], [], []);) 2025.09.25 21:12:08.154029 [ 2707 ] {c7500beb-ba9e-4922-91c3-de5f06fd2929} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt64OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toInt64OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:08.373972 [ 2707 ] {502af83c-3625-4926-b0de-00c8006a9739} executeQuery: Code: 42. DB::Exception: Number of arguments for function toInt64OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt64OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt64OrNull([], [], []);) 2025.09.25 21:12:08.381405 [ 2707 ] {502af83c-3625-4926-b0de-00c8006a9739} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toInt64OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt64OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:08.541267 [ 2707 ] {afee0c75-bb92-4e9c-9dda-9c88dc4f933f} executeQuery: Code: 42. DB::Exception: Number of arguments for function toInt64OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt64OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt64OrZero([], [], []);) 2025.09.25 21:12:08.547445 [ 2707 ] {afee0c75-bb92-4e9c-9dda-9c88dc4f933f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toInt64OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt64OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:08.625168 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:08.634050 [ 2707 ] {87f98614-39b4-4198-ac2f-d93e863253f7} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt8'. Expected 1 argument, got 3 arguments: In scope SELECT toInt8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt8([], [], []);) 2025.09.25 21:12:08.640006 [ 2707 ] {87f98614-39b4-4198-ac2f-d93e863253f7} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt8'. Expected 1 argument, got 3 arguments: In scope SELECT toInt8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:08.645478 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:08.795744 [ 2707 ] {fa1d50a9-b1cd-4e27-9446-ed51df902aa1} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt8OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toInt8OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt8OrDefault([], [], []);) 2025.09.25 21:12:08.821941 [ 2707 ] {fa1d50a9-b1cd-4e27-9446-ed51df902aa1} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toInt8OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toInt8OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:08.926570 [ 2707 ] {f3ec26f9-b082-4943-9af5-b91a1ece8b2c} executeQuery: Code: 42. DB::Exception: Number of arguments for function toInt8OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt8OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt8OrNull([], [], []);) 2025.09.25 21:12:08.931374 [ 2707 ] {f3ec26f9-b082-4943-9af5-b91a1ece8b2c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toInt8OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt8OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:09.076772 [ 2707 ] {886a3367-e118-433a-ae18-ae986cac0fb8} executeQuery: Code: 42. DB::Exception: Number of arguments for function toInt8OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt8OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toInt8OrZero([], [], []);) 2025.09.25 21:12:09.085411 [ 2707 ] {886a3367-e118-433a-ae18-ae986cac0fb8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toInt8OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toInt8OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:09.156451 [ 2707 ] {626abff8-a559-4c51-98c8-8337d3e9d5ac} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalDay'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalDay([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIntervalDay([], [], []);) 2025.09.25 21:12:09.162007 [ 2707 ] {626abff8-a559-4c51-98c8-8337d3e9d5ac} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalDay'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalDay([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:09.242046 [ 2707 ] {2db0e5d1-b9f9-4987-bacc-73e3e123a491} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalHour'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalHour([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIntervalHour([], [], []);) 2025.09.25 21:12:09.248809 [ 2707 ] {2db0e5d1-b9f9-4987-bacc-73e3e123a491} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalHour'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalHour([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:09.338260 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:09.347040 [ 2707 ] {c00c2e14-258e-4354-a9ef-0629f47e9158} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalMicrosecond'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalMicrosecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIntervalMicrosecond([], [], []);) 2025.09.25 21:12:09.354843 [ 2707 ] {c00c2e14-258e-4354-a9ef-0629f47e9158} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalMicrosecond'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalMicrosecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:09.363664 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:09.432578 [ 2707 ] {76764a69-a11d-4ac5-ab9c-a8704e45a492} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalMillisecond'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalMillisecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIntervalMillisecond([], [], []);) 2025.09.25 21:12:09.441165 [ 2707 ] {76764a69-a11d-4ac5-ab9c-a8704e45a492} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalMillisecond'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalMillisecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:09.504451 [ 2707 ] {25585072-7899-4114-b792-fb15b86939a4} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalMinute'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalMinute([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIntervalMinute([], [], []);) 2025.09.25 21:12:09.508702 [ 2707 ] {25585072-7899-4114-b792-fb15b86939a4} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalMinute'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalMinute([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:09.577912 [ 2707 ] {fbac1294-49f9-4adc-814c-64c3aa328780} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalMonth'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalMonth([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIntervalMonth([], [], []);) 2025.09.25 21:12:09.589399 [ 2707 ] {fbac1294-49f9-4adc-814c-64c3aa328780} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalMonth'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalMonth([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:09.646998 [ 2707 ] {1f6ecd68-e826-4d86-8441-6653da4f840b} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalNanosecond'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalNanosecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIntervalNanosecond([], [], []);) 2025.09.25 21:12:09.657692 [ 2707 ] {1f6ecd68-e826-4d86-8441-6653da4f840b} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalNanosecond'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalNanosecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:09.751241 [ 2707 ] {824f01bd-af53-4620-b103-4e96bc2177d2} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalQuarter'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalQuarter([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIntervalQuarter([], [], []);) 2025.09.25 21:12:09.755685 [ 2707 ] {824f01bd-af53-4620-b103-4e96bc2177d2} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalQuarter'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalQuarter([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:09.853864 [ 2707 ] {9cff9993-77b3-4201-906f-6b4886d0b144} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalSecond'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalSecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIntervalSecond([], [], []);) 2025.09.25 21:12:09.860092 [ 2707 ] {9cff9993-77b3-4201-906f-6b4886d0b144} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalSecond'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalSecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:09.909815 [ 2707 ] {991b6706-167f-4e8c-9f6e-180fc00c4930} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalWeek'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalWeek([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIntervalWeek([], [], []);) 2025.09.25 21:12:09.916172 [ 2707 ] {991b6706-167f-4e8c-9f6e-180fc00c4930} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalWeek'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalWeek([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:09.972315 [ 2707 ] {f009777b-d8ea-40d3-a0fe-730309617855} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalYear'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalYear([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toIntervalYear([], [], []);) 2025.09.25 21:12:09.978437 [ 2707 ] {f009777b-d8ea-40d3-a0fe-730309617855} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toIntervalYear'. Expected 1 argument, got 3 arguments: In scope SELECT toIntervalYear([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:10.046068 [ 2707 ] {95d00d8e-8d35-43d1-9b75-9d6fd5e3aa07} executeQuery: Code: 42. DB::Exception: Number of arguments for function toJSONString doesn't match: passed 3, should be 1: In scope SELECT toJSONString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toJSONString([], [], []);) 2025.09.25 21:12:10.051622 [ 2707 ] {95d00d8e-8d35-43d1-9b75-9d6fd5e3aa07} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toJSONString doesn't match: passed 3, should be 1: In scope SELECT toJSONString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:10.101884 [ 2707 ] {e36dba14-8efc-470d-a6b3-6a47d0a776ab} executeQuery: Code: 42. DB::Exception: Number of arguments for function toLastDayOfMonth doesn't match: passed 3, should be 1 or 2: In scope SELECT toLastDayOfMonth([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toLastDayOfMonth([], [], []);) 2025.09.25 21:12:10.107257 [ 2707 ] {e36dba14-8efc-470d-a6b3-6a47d0a776ab} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toLastDayOfMonth doesn't match: passed 3, should be 1 or 2: In scope SELECT toLastDayOfMonth([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:10.158592 [ 2707 ] {2f14ee84-1d7c-4476-88ff-eb7e31da4374} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function toLastDayOfWeek. Must be Date, Date32, DateTime or DateTime64: In scope SELECT toLastDayOfWeek([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toLastDayOfWeek([], [], []);) 2025.09.25 21:12:10.170776 [ 2707 ] {2f14ee84-1d7c-4476-88ff-eb7e31da4374} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function toLastDayOfWeek. Must be Date, Date32, DateTime or DateTime64: In scope SELECT toLastDayOfWeek([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:10.211763 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:10.277558 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:10.294930 [ 2707 ] {da939fc2-70d7-457d-aa13-a4253cd6d527} executeQuery: Code: 42. DB::Exception: Number of arguments for function toLowCardinality doesn't match: passed 3, should be 1: In scope SELECT toLowCardinality([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toLowCardinality([], [], []);) 2025.09.25 21:12:10.301455 [ 2707 ] {da939fc2-70d7-457d-aa13-a4253cd6d527} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toLowCardinality doesn't match: passed 3, should be 1: In scope SELECT toLowCardinality([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:10.450288 [ 2707 ] {61c40a2d-36fc-426c-b2d9-9ac6c05a2473} executeQuery: Code: 42. DB::Exception: Number of arguments for function toMillisecond doesn't match: passed 3, should be 1 or 2: In scope SELECT toMillisecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toMillisecond([], [], []);) 2025.09.25 21:12:10.466771 [ 2707 ] {61c40a2d-36fc-426c-b2d9-9ac6c05a2473} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toMillisecond doesn't match: passed 3, should be 1 or 2: In scope SELECT toMillisecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:10.550234 [ 2707 ] {b4398263-ffac-4b6a-b6fb-61e84737524c} executeQuery: Code: 42. DB::Exception: Number of arguments for function toMinute doesn't match: passed 3, should be 1 or 2: In scope SELECT toMinute([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toMinute([], [], []);) 2025.09.25 21:12:10.556201 [ 2707 ] {b4398263-ffac-4b6a-b6fb-61e84737524c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toMinute doesn't match: passed 3, should be 1 or 2: In scope SELECT toMinute([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:10.653356 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:10.675859 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:10.751770 [ 2707 ] {946fb065-1f51-47a1-a448-bad69cecbc7b} executeQuery: Code: 42. DB::Exception: Number of arguments for function toModifiedJulianDay doesn't match: passed 3, should be 1: In scope SELECT toModifiedJulianDay([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toModifiedJulianDay([], [], []);) 2025.09.25 21:12:10.755949 [ 2707 ] {946fb065-1f51-47a1-a448-bad69cecbc7b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toModifiedJulianDay doesn't match: passed 3, should be 1: In scope SELECT toModifiedJulianDay([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:10.802290 [ 2707 ] {005d13c4-cd46-402b-b7b5-654fa4528043} executeQuery: Code: 42. DB::Exception: Number of arguments for function toModifiedJulianDayOrNull doesn't match: passed 3, should be 1: In scope SELECT toModifiedJulianDayOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toModifiedJulianDayOrNull([], [], []);) 2025.09.25 21:12:10.807757 [ 2707 ] {005d13c4-cd46-402b-b7b5-654fa4528043} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toModifiedJulianDayOrNull doesn't match: passed 3, should be 1: In scope SELECT toModifiedJulianDayOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:10.921537 [ 2707 ] {81d331f7-a5c3-4c92-a45f-a7ca3ec1a5ab} executeQuery: Code: 42. DB::Exception: Number of arguments for function toMonday doesn't match: passed 3, should be 1 or 2: In scope SELECT toMonday([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toMonday([], [], []);) 2025.09.25 21:12:10.929563 [ 2707 ] {81d331f7-a5c3-4c92-a45f-a7ca3ec1a5ab} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toMonday doesn't match: passed 3, should be 1 or 2: In scope SELECT toMonday([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:11.089754 [ 2707 ] {b5d49901-6c78-4ef9-8755-83625d0ab9ab} executeQuery: Code: 42. DB::Exception: Number of arguments for function toMonth doesn't match: passed 3, should be 1 or 2: In scope SELECT toMonth([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toMonth([], [], []);) 2025.09.25 21:12:11.103825 [ 2707 ] {b5d49901-6c78-4ef9-8755-83625d0ab9ab} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toMonth doesn't match: passed 3, should be 1 or 2: In scope SELECT toMonth([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:11.201576 [ 2707 ] {8dff868b-aca3-4784-a533-5f159cdfefa1} executeQuery: Code: 42. DB::Exception: Number of arguments for function toNullable doesn't match: passed 3, should be 1: In scope SELECT toNullable([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toNullable([], [], []);) 2025.09.25 21:12:11.216351 [ 2707 ] {8dff868b-aca3-4784-a533-5f159cdfefa1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toNullable doesn't match: passed 3, should be 1: In scope SELECT toNullable([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:11.351454 [ 2707 ] {3a0a5858-a753-4fa1-8e93-bf439dcd0176} executeQuery: Code: 42. DB::Exception: Number of arguments for function toQuarter doesn't match: passed 3, should be 1 or 2: In scope SELECT toQuarter([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toQuarter([], [], []);) 2025.09.25 21:12:11.359753 [ 2707 ] {3a0a5858-a753-4fa1-8e93-bf439dcd0176} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toQuarter doesn't match: passed 3, should be 1 or 2: In scope SELECT toQuarter([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:11.463865 [ 2707 ] {388cd250-f057-4aa3-a4f7-b267d1116d2a} executeQuery: Code: 42. DB::Exception: Number of arguments for function toRelativeDayNum doesn't match: passed 3, should be 1 or 2: In scope SELECT toRelativeDayNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toRelativeDayNum([], [], []);) 2025.09.25 21:12:11.472760 [ 2707 ] {388cd250-f057-4aa3-a4f7-b267d1116d2a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toRelativeDayNum doesn't match: passed 3, should be 1 or 2: In scope SELECT toRelativeDayNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:11.516131 [ 3619 ] {3d424325-5c32-4886-953d-e4e6d23f30a4} executeQuery: Code: 452. DB::Exception: Setting allow_experimental_object_type should not be changed. (SETTING_CONSTRAINT_VIOLATION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43966) (comment: 01825_type_json_1.sql) (in query: SET allow_experimental_object_type = 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges const&, DB::SettingSource) const @ 0x0000000026f807e3 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraints(DB::SettingsChanges const&, DB::SettingSource) @ 0x000000002853b6ad 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::execute() @ 0x00000000294f603b 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:11.520731 [ 3619 ] {3d424325-5c32-4886-953d-e4e6d23f30a4} TCPHandler: Code: 452. DB::Exception: Setting allow_experimental_object_type should not be changed. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges const&, DB::SettingSource) const @ 0x0000000026f807e3 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraints(DB::SettingsChanges const&, DB::SettingSource) @ 0x000000002853b6ad 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::execute() @ 0x00000000294f603b 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:11.549072 [ 2707 ] {ef9e1d16-8d96-46be-a5ec-ecc5bf88f559} executeQuery: Code: 42. DB::Exception: Number of arguments for function toRelativeHourNum doesn't match: passed 3, should be 1 or 2: In scope SELECT toRelativeHourNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toRelativeHourNum([], [], []);) 2025.09.25 21:12:11.563752 [ 2707 ] {ef9e1d16-8d96-46be-a5ec-ecc5bf88f559} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toRelativeHourNum doesn't match: passed 3, should be 1 or 2: In scope SELECT toRelativeHourNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:11.574536 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:11.603853 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:11.640651 [ 2707 ] {6a047ed7-88c2-4c8e-9196-6038ee404ff6} executeQuery: Code: 42. DB::Exception: Number of arguments for function toRelativeMinuteNum doesn't match: passed 3, should be 1 or 2: In scope SELECT toRelativeMinuteNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toRelativeMinuteNum([], [], []);) 2025.09.25 21:12:11.671789 [ 2707 ] {6a047ed7-88c2-4c8e-9196-6038ee404ff6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toRelativeMinuteNum doesn't match: passed 3, should be 1 or 2: In scope SELECT toRelativeMinuteNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:11.732497 [ 2707 ] {f274c162-dd82-4308-b148-aa8d37aa82d7} executeQuery: Code: 42. DB::Exception: Number of arguments for function toRelativeMonthNum doesn't match: passed 3, should be 1 or 2: In scope SELECT toRelativeMonthNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toRelativeMonthNum([], [], []);) 2025.09.25 21:12:11.738817 [ 2707 ] {f274c162-dd82-4308-b148-aa8d37aa82d7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toRelativeMonthNum doesn't match: passed 3, should be 1 or 2: In scope SELECT toRelativeMonthNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:11.790437 [ 2707 ] {d7a9492b-fbb0-4a06-b7e9-602f191c411b} executeQuery: Code: 42. DB::Exception: Number of arguments for function toRelativeQuarterNum doesn't match: passed 3, should be 1 or 2: In scope SELECT toRelativeQuarterNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toRelativeQuarterNum([], [], []);) 2025.09.25 21:12:11.795902 [ 2707 ] {d7a9492b-fbb0-4a06-b7e9-602f191c411b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toRelativeQuarterNum doesn't match: passed 3, should be 1 or 2: In scope SELECT toRelativeQuarterNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:11.850132 [ 2707 ] {3f7681f6-7a27-446d-a267-2a0c4042883c} executeQuery: Code: 42. DB::Exception: Number of arguments for function toRelativeSecondNum doesn't match: passed 3, should be 1 or 2: In scope SELECT toRelativeSecondNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toRelativeSecondNum([], [], []);) 2025.09.25 21:12:11.917248 [ 2707 ] {3f7681f6-7a27-446d-a267-2a0c4042883c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toRelativeSecondNum doesn't match: passed 3, should be 1 or 2: In scope SELECT toRelativeSecondNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:12.094257 [ 2707 ] {9f906911-c3ef-44a3-94c8-d62baf63672f} executeQuery: Code: 42. DB::Exception: Number of arguments for function toRelativeWeekNum doesn't match: passed 3, should be 1 or 2: In scope SELECT toRelativeWeekNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toRelativeWeekNum([], [], []);) 2025.09.25 21:12:12.104192 [ 2707 ] {9f906911-c3ef-44a3-94c8-d62baf63672f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toRelativeWeekNum doesn't match: passed 3, should be 1 or 2: In scope SELECT toRelativeWeekNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:12.184736 [ 2707 ] {06d3e1a9-f296-404a-96e3-0b010d6b63d4} executeQuery: Code: 42. DB::Exception: Number of arguments for function toRelativeYearNum doesn't match: passed 3, should be 1 or 2: In scope SELECT toRelativeYearNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toRelativeYearNum([], [], []);) 2025.09.25 21:12:12.201445 [ 2707 ] {06d3e1a9-f296-404a-96e3-0b010d6b63d4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toRelativeYearNum doesn't match: passed 3, should be 1 or 2: In scope SELECT toRelativeYearNum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:12.323415 [ 2707 ] {9eb3ee75-f463-44c6-b834-d1e8b5abae40} executeQuery: Code: 42. DB::Exception: Number of arguments for function toSecond doesn't match: passed 3, should be 1 or 2: In scope SELECT toSecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toSecond([], [], []);) 2025.09.25 21:12:12.330353 [ 2707 ] {9eb3ee75-f463-44c6-b834-d1e8b5abae40} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toSecond doesn't match: passed 3, should be 1 or 2: In scope SELECT toSecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:12.421019 [ 2707 ] {cd84e309-c9a9-4765-ae1a-4db84e993e84} executeQuery: Code: 42. DB::Exception: Number of arguments for function toStartOfDay doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfDay([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfDay([], [], []);) 2025.09.25 21:12:12.429993 [ 2707 ] {cd84e309-c9a9-4765-ae1a-4db84e993e84} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStartOfDay doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfDay([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:12.466357 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:12.497752 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:12.516046 [ 2707 ] {71cda1ae-3aeb-4b5a-835f-5962d048e756} executeQuery: Code: 42. DB::Exception: Number of arguments for function toStartOfFifteenMinutes doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfFifteenMinutes([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfFifteenMinutes([], [], []);) 2025.09.25 21:12:12.521473 [ 2707 ] {71cda1ae-3aeb-4b5a-835f-5962d048e756} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStartOfFifteenMinutes doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfFifteenMinutes([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:12.629015 [ 2707 ] {802dd5db-3bc3-4ee4-b727-76d9800431ae} executeQuery: Code: 42. DB::Exception: Number of arguments for function toStartOfFiveMinutes doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfFiveMinute([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfFiveMinute([], [], []);) 2025.09.25 21:12:12.634394 [ 2707 ] {802dd5db-3bc3-4ee4-b727-76d9800431ae} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStartOfFiveMinutes doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfFiveMinute([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:12.693332 [ 2707 ] {589b7b2e-a4d9-4304-91be-45658dac8348} executeQuery: Code: 42. DB::Exception: Number of arguments for function toStartOfFiveMinutes doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfFiveMinutes([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfFiveMinutes([], [], []);) 2025.09.25 21:12:12.700200 [ 2707 ] {589b7b2e-a4d9-4304-91be-45658dac8348} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStartOfFiveMinutes doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfFiveMinutes([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:12.758783 [ 2707 ] {b2aa8a25-2ed7-4062-8e18-9cbd5f858760} executeQuery: Code: 42. DB::Exception: Number of arguments for function toStartOfHour doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfHour([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfHour([], [], []);) 2025.09.25 21:12:12.763901 [ 2707 ] {b2aa8a25-2ed7-4062-8e18-9cbd5f858760} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStartOfHour doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfHour([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:12.819221 [ 2707 ] {235199b6-f8c1-43cf-9f01-49e07f5872ee} executeQuery: Code: 42. DB::Exception: Number of arguments for function toStartOfISOYear doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfISOYear([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfISOYear([], [], []);) 2025.09.25 21:12:12.825440 [ 2707 ] {235199b6-f8c1-43cf-9f01-49e07f5872ee} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStartOfISOYear doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfISOYear([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:13.052682 [ 2707 ] {a172cfd6-b2e3-4a2c-9923-31d4610c35e4} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of 1st argument of function toStartOfInterval, expected a Date, DateTime or DateTime64: In scope SELECT toStartOfInterval([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfInterval([], [], []);) 2025.09.25 21:12:13.058343 [ 2707 ] {a172cfd6-b2e3-4a2c-9923-31d4610c35e4} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of 1st argument of function toStartOfInterval, expected a Date, DateTime or DateTime64: In scope SELECT toStartOfInterval([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:13.115411 [ 2707 ] {777d313e-af9e-4286-bf10-d0779f44c966} executeQuery: Code: 42. DB::Exception: Number of arguments for function toStartOfMicrosecond doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfMicrosecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfMicrosecond([], [], []);) 2025.09.25 21:12:13.127680 [ 2707 ] {777d313e-af9e-4286-bf10-d0779f44c966} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStartOfMicrosecond doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfMicrosecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:13.192574 [ 2707 ] {c1c529c6-3d39-43f1-ac44-35cddf69b40d} executeQuery: Code: 42. DB::Exception: Number of arguments for function toStartOfMillisecond doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfMillisecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfMillisecond([], [], []);) 2025.09.25 21:12:13.199125 [ 2707 ] {c1c529c6-3d39-43f1-ac44-35cddf69b40d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStartOfMillisecond doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfMillisecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:13.264611 [ 2707 ] {36f4c6e3-3e09-424f-a8c8-b0d18d1869b4} executeQuery: Code: 42. DB::Exception: Number of arguments for function toStartOfMinute doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfMinute([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfMinute([], [], []);) 2025.09.25 21:12:13.269556 [ 2707 ] {36f4c6e3-3e09-424f-a8c8-b0d18d1869b4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStartOfMinute doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfMinute([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:13.399251 [ 2707 ] {d1869a39-b9e0-4858-ae84-590a38f553ac} executeQuery: Code: 42. DB::Exception: Number of arguments for function toStartOfMonth doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfMonth([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfMonth([], [], []);) 2025.09.25 21:12:13.404456 [ 2707 ] {d1869a39-b9e0-4858-ae84-590a38f553ac} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStartOfMonth doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfMonth([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:13.439462 [ 2707 ] {dcd99344-c60a-4957-83ad-23c5360e0bce} executeQuery: Code: 42. DB::Exception: Number of arguments for function toStartOfNanosecond doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfNanosecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfNanosecond([], [], []);) 2025.09.25 21:12:13.443696 [ 2707 ] {dcd99344-c60a-4957-83ad-23c5360e0bce} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStartOfNanosecond doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfNanosecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:13.450270 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:13.459063 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:13.491628 [ 2707 ] {d09d50c8-494d-483f-bf68-efb232e5b9cc} executeQuery: Code: 42. DB::Exception: Number of arguments for function toStartOfQuarter doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfQuarter([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfQuarter([], [], []);) 2025.09.25 21:12:13.494806 [ 2707 ] {d09d50c8-494d-483f-bf68-efb232e5b9cc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStartOfQuarter doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfQuarter([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:13.534835 [ 2707 ] {483eb932-3444-449b-bb15-cca917bdbb41} executeQuery: Code: 42. DB::Exception: Number of arguments for function toStartOfSecond doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfSecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfSecond([], [], []);) 2025.09.25 21:12:13.538029 [ 2707 ] {483eb932-3444-449b-bb15-cca917bdbb41} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStartOfSecond doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfSecond([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:13.619473 [ 2707 ] {f1d6b9d1-eaa3-4afb-92b5-f56d0a906ef7} executeQuery: Code: 42. DB::Exception: Number of arguments for function toStartOfTenMinutes doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfTenMinutes([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfTenMinutes([], [], []);) 2025.09.25 21:12:13.624578 [ 2707 ] {f1d6b9d1-eaa3-4afb-92b5-f56d0a906ef7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStartOfTenMinutes doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfTenMinutes([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:13.683628 [ 2707 ] {ecb93ae5-f55d-416b-967d-91e63afbdb96} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function toStartOfWeek. Must be Date, Date32, DateTime or DateTime64: In scope SELECT toStartOfWeek([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfWeek([], [], []);) 2025.09.25 21:12:13.698644 [ 2707 ] {ecb93ae5-f55d-416b-967d-91e63afbdb96} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function toStartOfWeek. Must be Date, Date32, DateTime or DateTime64: In scope SELECT toStartOfWeek([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:13.749102 [ 2707 ] {0ab67c02-6671-4798-b2b4-543cf5ba2501} executeQuery: Code: 42. DB::Exception: Number of arguments for function toStartOfYear doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfYear([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStartOfYear([], [], []);) 2025.09.25 21:12:13.758755 [ 2707 ] {0ab67c02-6671-4798-b2b4-543cf5ba2501} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStartOfYear doesn't match: passed 3, should be 1 or 2: In scope SELECT toStartOfYear([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:13.808199 [ 2707 ] {499602b6-f71b-4b0c-a123-087315934b46} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toString'. Expected 1 argument, got 3 arguments: In scope SELECT toString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toString([], [], []);) 2025.09.25 21:12:13.813682 [ 2707 ] {499602b6-f71b-4b0c-a123-087315934b46} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toString'. Expected 1 argument, got 3 arguments: In scope SELECT toString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:13.853346 [ 2707 ] {cc4a0c3a-5ea0-456c-8a35-2798096ace2e} executeQuery: Code: 42. DB::Exception: Number of arguments for function toStringCutToZero doesn't match: passed 3, should be 1: In scope SELECT toStringCutToZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toStringCutToZero([], [], []);) 2025.09.25 21:12:13.859024 [ 2707 ] {cc4a0c3a-5ea0-456c-8a35-2798096ace2e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toStringCutToZero doesn't match: passed 3, should be 1: In scope SELECT toStringCutToZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:13.866590 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:13.905695 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:13.957055 [ 2707 ] {e7dc6ba0-a24f-4223-ae55-697b995edf3b} executeQuery: Code: 42. DB::Exception: Number of arguments for function toTime doesn't match: passed 3, should be 1 or 2: In scope SELECT toTime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toTime([], [], []);) 2025.09.25 21:12:13.962231 [ 2707 ] {e7dc6ba0-a24f-4223-ae55-697b995edf3b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toTime doesn't match: passed 3, should be 1 or 2: In scope SELECT toTime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:14.014423 [ 2707 ] {aacf9720-5371-4e1b-b3e7-0d916d5507ed} executeQuery: Code: 42. DB::Exception: Number of arguments for function toTimezone doesn't match: passed 3, should be 2: In scope SELECT toTimeZone([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toTimeZone([], [], []);) 2025.09.25 21:12:14.019603 [ 2707 ] {aacf9720-5371-4e1b-b3e7-0d916d5507ed} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toTimezone doesn't match: passed 3, should be 2: In scope SELECT toTimeZone([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:14.059424 [ 2707 ] {77cf68bf-8ecc-4e44-b02c-5beb399dc736} executeQuery: Code: 42. DB::Exception: Number of arguments for function toTimezone doesn't match: passed 3, should be 2: In scope SELECT toTimezone([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toTimezone([], [], []);) 2025.09.25 21:12:14.065306 [ 2707 ] {77cf68bf-8ecc-4e44-b02c-5beb399dc736} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toTimezone doesn't match: passed 3, should be 2: In scope SELECT toTimezone([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:14.109509 [ 2707 ] {a535f595-89ea-4aca-8eb0-a971b82f4efd} executeQuery: Code: 42. DB::Exception: Number of arguments for function toTypeName doesn't match: passed 3, should be 1: In scope SELECT toTypeName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toTypeName([], [], []);) 2025.09.25 21:12:14.112999 [ 2707 ] {a535f595-89ea-4aca-8eb0-a971b82f4efd} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toTypeName doesn't match: passed 3, should be 1: In scope SELECT toTypeName([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:14.176699 [ 2707 ] {a83bdb5a-2ed0-4192-9955-d2d2991fd471} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt128'. Expected 1 argument, got 3 arguments: In scope SELECT toUInt128([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt128([], [], []);) 2025.09.25 21:12:14.181322 [ 2707 ] {a83bdb5a-2ed0-4192-9955-d2d2991fd471} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt128'. Expected 1 argument, got 3 arguments: In scope SELECT toUInt128([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:14.233023 [ 2707 ] {ce592213-c813-427e-b034-748b496e6074} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt128OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toUInt128OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt128OrDefault([], [], []);) 2025.09.25 21:12:14.238296 [ 2707 ] {ce592213-c813-427e-b034-748b496e6074} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt128OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toUInt128OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:14.295620 [ 2707 ] {3476bad5-1f8e-4259-bf6f-7457e42fc73a} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUInt128OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt128OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt128OrNull([], [], []);) 2025.09.25 21:12:14.300784 [ 2707 ] {3476bad5-1f8e-4259-bf6f-7457e42fc73a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUInt128OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt128OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:14.445481 [ 2707 ] {b5389f00-dcaa-4488-be93-b55851e3d204} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUInt128OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt128OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt128OrZero([], [], []);) 2025.09.25 21:12:14.450287 [ 2707 ] {b5389f00-dcaa-4488-be93-b55851e3d204} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUInt128OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt128OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:14.507953 [ 2707 ] {f2420a6e-6e7d-4bad-a763-b0d7a50958a6} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt16'. Expected 1 argument, got 3 arguments: In scope SELECT toUInt16([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt16([], [], []);) 2025.09.25 21:12:14.513392 [ 2707 ] {f2420a6e-6e7d-4bad-a763-b0d7a50958a6} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt16'. Expected 1 argument, got 3 arguments: In scope SELECT toUInt16([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:14.628661 [ 2707 ] {cbd6fae5-ac41-445d-a7b6-678a30bc21a5} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt16OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toUInt16OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt16OrDefault([], [], []);) 2025.09.25 21:12:14.632785 [ 2707 ] {cbd6fae5-ac41-445d-a7b6-678a30bc21a5} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt16OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toUInt16OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:14.689993 [ 2707 ] {fdb1a598-298f-463f-8ead-981c79a3ecc2} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUInt16OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt16OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt16OrNull([], [], []);) 2025.09.25 21:12:14.695552 [ 2707 ] {fdb1a598-298f-463f-8ead-981c79a3ecc2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUInt16OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt16OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:14.749715 [ 2707 ] {044dd239-80c4-4a0a-9f72-1c25cd777524} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUInt16OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt16OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt16OrZero([], [], []);) 2025.09.25 21:12:14.754704 [ 2707 ] {044dd239-80c4-4a0a-9f72-1c25cd777524} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUInt16OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt16OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:14.854287 [ 2707 ] {9d25ba81-c97f-4da0-91d7-ba5774888706} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt256'. Expected 1 argument, got 3 arguments: In scope SELECT toUInt256([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt256([], [], []);) 2025.09.25 21:12:14.861143 [ 2707 ] {9d25ba81-c97f-4da0-91d7-ba5774888706} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt256'. Expected 1 argument, got 3 arguments: In scope SELECT toUInt256([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:14.911157 [ 2707 ] {bdf64248-0a4d-4581-8f99-ad7ff28406dc} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt256OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toUInt256OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt256OrDefault([], [], []);) 2025.09.25 21:12:14.916187 [ 2707 ] {bdf64248-0a4d-4581-8f99-ad7ff28406dc} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt256OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toUInt256OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:14.972851 [ 2707 ] {3e0ae5bf-eb69-46a9-9f66-c0319afeda8b} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUInt256OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt256OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt256OrNull([], [], []);) 2025.09.25 21:12:14.978281 [ 2707 ] {3e0ae5bf-eb69-46a9-9f66-c0319afeda8b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUInt256OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt256OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:15.040259 [ 2707 ] {afb9d4ac-3968-4ffb-9dde-646a31fc13b1} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUInt256OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt256OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt256OrZero([], [], []);) 2025.09.25 21:12:15.049989 [ 2707 ] {afb9d4ac-3968-4ffb-9dde-646a31fc13b1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUInt256OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt256OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:15.144952 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:15.154248 [ 2707 ] {066bda5e-0749-48d1-a33d-4ec82481d845} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt32'. Expected 1 argument, got 3 arguments: In scope SELECT toUInt32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt32([], [], []);) 2025.09.25 21:12:15.158399 [ 2707 ] {066bda5e-0749-48d1-a33d-4ec82481d845} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt32'. Expected 1 argument, got 3 arguments: In scope SELECT toUInt32([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:15.162991 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:15.216756 [ 2707 ] {efb88f16-3fcd-4ff4-88d9-d612342d8a14} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt32OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toUInt32OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt32OrDefault([], [], []);) 2025.09.25 21:12:15.222602 [ 2707 ] {efb88f16-3fcd-4ff4-88d9-d612342d8a14} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt32OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toUInt32OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:15.275756 [ 2707 ] {51a73b41-f700-4ff7-b029-2101c74fb508} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUInt32OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt32OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt32OrNull([], [], []);) 2025.09.25 21:12:15.280761 [ 2707 ] {51a73b41-f700-4ff7-b029-2101c74fb508} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUInt32OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt32OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:15.361656 [ 2707 ] {1b7d8041-a369-4e68-b548-d7a643370fa7} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUInt32OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt32OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt32OrZero([], [], []);) 2025.09.25 21:12:15.367202 [ 2707 ] {1b7d8041-a369-4e68-b548-d7a643370fa7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUInt32OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt32OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:15.419403 [ 2707 ] {da522b2b-6494-4325-bf58-4203e3b7b6c5} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt64'. Expected 1 argument, got 3 arguments: In scope SELECT toUInt64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt64([], [], []);) 2025.09.25 21:12:15.431063 [ 2707 ] {da522b2b-6494-4325-bf58-4203e3b7b6c5} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt64'. Expected 1 argument, got 3 arguments: In scope SELECT toUInt64([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:15.563987 [ 2707 ] {e892b19a-2ae0-4d41-8c81-2a0c5b8360ea} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt64OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toUInt64OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt64OrDefault([], [], []);) 2025.09.25 21:12:15.569912 [ 2707 ] {e892b19a-2ae0-4d41-8c81-2a0c5b8360ea} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt64OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toUInt64OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:15.627757 [ 2707 ] {5f8372fe-31e2-4e77-bcd1-0e31bf63f897} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUInt64OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt64OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt64OrNull([], [], []);) 2025.09.25 21:12:15.636145 [ 2707 ] {5f8372fe-31e2-4e77-bcd1-0e31bf63f897} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUInt64OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt64OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:15.709897 [ 2707 ] {528b1a28-f9e2-43ec-9276-0b5e328cfbc4} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUInt64OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt64OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt64OrZero([], [], []);) 2025.09.25 21:12:15.714449 [ 2707 ] {528b1a28-f9e2-43ec-9276-0b5e328cfbc4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUInt64OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt64OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:15.806789 [ 2707 ] {c81fadd8-8670-4850-a97d-a70e902c8537} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt8'. Expected 1 argument, got 3 arguments: In scope SELECT toUInt8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt8([], [], []);) 2025.09.25 21:12:15.811959 [ 2707 ] {c81fadd8-8670-4850-a97d-a70e902c8537} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt8'. Expected 1 argument, got 3 arguments: In scope SELECT toUInt8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:15.875657 [ 2707 ] {e32b4855-d13c-43b4-963b-cbf0f974ad1e} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt8OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toUInt8OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt8OrDefault([], [], []);) 2025.09.25 21:12:15.882150 [ 2707 ] {e32b4855-d13c-43b4-963b-cbf0f974ad1e} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUInt8OrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toUInt8OrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:15.908621 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:15.976058 [ 2707 ] {0f3f3d6a-fd08-4af6-ad80-cd6f38afc29e} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUInt8OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt8OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt8OrNull([], [], []);) 2025.09.25 21:12:15.993735 [ 2707 ] {0f3f3d6a-fd08-4af6-ad80-cd6f38afc29e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUInt8OrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt8OrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:16.018274 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:16.067103 [ 2707 ] {352126a9-979c-4162-a773-e7b39ae31b9f} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUInt8OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt8OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUInt8OrZero([], [], []);) 2025.09.25 21:12:16.072815 [ 2707 ] {352126a9-979c-4162-a773-e7b39ae31b9f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUInt8OrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUInt8OrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:16.158632 [ 2707 ] {1c8e37ae-2100-4b17-8460-4913b918b08e} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUTCTimestamp doesn't match: passed 3, should be 2: In scope SELECT toUTCTimestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUTCTimestamp([], [], []);) 2025.09.25 21:12:16.166407 [ 2707 ] {1c8e37ae-2100-4b17-8460-4913b918b08e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUTCTimestamp doesn't match: passed 3, should be 2: In scope SELECT toUTCTimestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:16.242547 [ 2707 ] {14fa87c4-adab-488a-bf69-1cc77d74fbd1} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUUID'. Expected 1 argument, got 3 arguments: In scope SELECT toUUID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUUID([], [], []);) 2025.09.25 21:12:16.248312 [ 2707 ] {14fa87c4-adab-488a-bf69-1cc77d74fbd1} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUUID'. Expected 1 argument, got 3 arguments: In scope SELECT toUUID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:16.302364 [ 2707 ] {2e855e2b-3ad3-4672-a780-3b05d368105f} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUUIDOrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toUUIDOrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUUIDOrDefault([], [], []);) 2025.09.25 21:12:16.308867 [ 2707 ] {2e855e2b-3ad3-4672-a780-3b05d368105f} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUUIDOrDefault'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toUUIDOrDefault([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:16.438631 [ 2707 ] {36d5a28f-1fca-4cd8-b1ca-82b31b62eecf} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUUIDOrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUUIDOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUUIDOrNull([], [], []);) 2025.09.25 21:12:16.444437 [ 2707 ] {36d5a28f-1fca-4cd8-b1ca-82b31b62eecf} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUUIDOrNull doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUUIDOrNull([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:16.610466 [ 2707 ] {c470f75f-fecf-4dd4-acce-654467346037} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUUIDOrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUUIDOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUUIDOrZero([], [], []);) 2025.09.25 21:12:16.615905 [ 2707 ] {c470f75f-fecf-4dd4-acce-654467346037} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUUIDOrZero doesn't match: passed 3, should be 1 or 2. Second argument only make sense for DateTime (time zone, optional) and Decimal (scale).: In scope SELECT toUUIDOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:16.763074 [ 2707 ] {156408ad-468d-42f7-911f-4c382519aca7} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUnixTimestamp'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toUnixTimestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUnixTimestamp([], [], []);) 2025.09.25 21:12:16.767334 [ 2707 ] {156408ad-468d-42f7-911f-4c382519aca7} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'toUnixTimestamp'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT toUnixTimestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:16.825127 [ 2707 ] {6bc66019-bac2-4a17-adf1-bf7f683e3ae3} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUnixTimestamp64Micro doesn't match: passed 3, should be 1: In scope SELECT toUnixTimestamp64Micro([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUnixTimestamp64Micro([], [], []);) 2025.09.25 21:12:16.831943 [ 2707 ] {6bc66019-bac2-4a17-adf1-bf7f683e3ae3} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUnixTimestamp64Micro doesn't match: passed 3, should be 1: In scope SELECT toUnixTimestamp64Micro([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:16.917486 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:16.935463 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:16.971690 [ 2707 ] {4062f67b-0782-4a12-b978-3aa2c7174423} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUnixTimestamp64Milli doesn't match: passed 3, should be 1: In scope SELECT toUnixTimestamp64Milli([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUnixTimestamp64Milli([], [], []);) 2025.09.25 21:12:16.979266 [ 2707 ] {4062f67b-0782-4a12-b978-3aa2c7174423} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUnixTimestamp64Milli doesn't match: passed 3, should be 1: In scope SELECT toUnixTimestamp64Milli([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:17.033137 [ 2707 ] {c16a04a1-8c47-48ca-bd3a-e7d1769d59ce} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUnixTimestamp64Nano doesn't match: passed 3, should be 1: In scope SELECT toUnixTimestamp64Nano([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toUnixTimestamp64Nano([], [], []);) 2025.09.25 21:12:17.035992 [ 2707 ] {c16a04a1-8c47-48ca-bd3a-e7d1769d59ce} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUnixTimestamp64Nano doesn't match: passed 3, should be 1: In scope SELECT toUnixTimestamp64Nano([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:17.140927 [ 2707 ] {6978962a-92fc-425b-ad7c-5532257312ad} executeQuery: Code: 42. DB::Exception: Number of arguments for function toValidUTF8 doesn't match: passed 3, should be 1: In scope SELECT toValidUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toValidUTF8([], [], []);) 2025.09.25 21:12:17.145704 [ 2707 ] {6978962a-92fc-425b-ad7c-5532257312ad} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toValidUTF8 doesn't match: passed 3, should be 1: In scope SELECT toValidUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:17.267336 [ 2707 ] {c3b9a2b1-d304-42fd-b946-498f774c761a} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function toWeek. Must be Date, Date32, DateTime or DateTime64: In scope SELECT toWeek([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toWeek([], [], []);) 2025.09.25 21:12:17.275034 [ 2707 ] {c3b9a2b1-d304-42fd-b946-498f774c761a} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function toWeek. Must be Date, Date32, DateTime or DateTime64: In scope SELECT toWeek([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:17.329269 [ 2707 ] {f0a71f5d-b2da-4bf5-9f3f-16bfa5436ee4} executeQuery: Code: 42. DB::Exception: Number of arguments for function toYYYYMM doesn't match: passed 3, should be 1 or 2: In scope SELECT toYYYYMM([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toYYYYMM([], [], []);) 2025.09.25 21:12:17.335222 [ 2707 ] {f0a71f5d-b2da-4bf5-9f3f-16bfa5436ee4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toYYYYMM doesn't match: passed 3, should be 1 or 2: In scope SELECT toYYYYMM([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:17.386513 [ 2707 ] {b23a471e-5292-492b-b21b-4d26ea7a89dd} executeQuery: Code: 42. DB::Exception: Number of arguments for function toYYYYMMDD doesn't match: passed 3, should be 1 or 2: In scope SELECT toYYYYMMDD([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toYYYYMMDD([], [], []);) 2025.09.25 21:12:17.392673 [ 2707 ] {b23a471e-5292-492b-b21b-4d26ea7a89dd} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toYYYYMMDD doesn't match: passed 3, should be 1 or 2: In scope SELECT toYYYYMMDD([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:17.469161 [ 2707 ] {9f881e8d-3be4-4a84-8586-f71d3fb04f33} executeQuery: Code: 42. DB::Exception: Number of arguments for function toYYYYMMDDhhmmss doesn't match: passed 3, should be 1 or 2: In scope SELECT toYYYYMMDDhhmmss([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toYYYYMMDDhhmmss([], [], []);) 2025.09.25 21:12:17.478427 [ 2707 ] {9f881e8d-3be4-4a84-8586-f71d3fb04f33} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toYYYYMMDDhhmmss doesn't match: passed 3, should be 1 or 2: In scope SELECT toYYYYMMDDhhmmss([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:17.557186 [ 2707 ] {03bfadb2-2d6b-46c9-b179-55a12c50909a} executeQuery: Code: 42. DB::Exception: Number of arguments for function toYear doesn't match: passed 3, should be 1 or 2: In scope SELECT toYear([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toYear([], [], []);) 2025.09.25 21:12:17.567896 [ 2707 ] {03bfadb2-2d6b-46c9-b179-55a12c50909a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toYear doesn't match: passed 3, should be 1 or 2: In scope SELECT toYear([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:17.672593 [ 2707 ] {5c4d5ca5-aa16-437a-87c4-95252d0d232d} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function toYearWeek. Must be Date, Date32, DateTime or DateTime64: In scope SELECT toYearWeek([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT toYearWeek([], [], []);) 2025.09.25 21:12:17.681925 [ 2707 ] {5c4d5ca5-aa16-437a-87c4-95252d0d232d} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function toYearWeek. Must be Date, Date32, DateTime or DateTime64: In scope SELECT toYearWeek([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:17.715534 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:17.733956 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:17.758098 [ 2707 ] {ac983ed9-c4a9-43b6-b461-66b1c8595a93} executeQuery: Code: 42. DB::Exception: Number of arguments for function toUTCTimestamp doesn't match: passed 3, should be 2: In scope SELECT to_utc_timestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT to_utc_timestamp([], [], []);) 2025.09.25 21:12:17.767712 [ 2707 ] {ac983ed9-c4a9-43b6-b461-66b1c8595a93} TCPHandler: Code: 42. DB::Exception: Number of arguments for function toUTCTimestamp doesn't match: passed 3, should be 2: In scope SELECT to_utc_timestamp([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:17.907136 [ 2707 ] {890b316c-9224-4579-bfd6-822d857982b1} executeQuery: Code: 42. DB::Exception: Number of arguments for function today doesn't match: passed 3, should be 0: In scope SELECT today([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT today([], [], []);) 2025.09.25 21:12:17.912484 [ 2707 ] {890b316c-9224-4579-bfd6-822d857982b1} TCPHandler: Code: 42. DB::Exception: Number of arguments for function today doesn't match: passed 3, should be 0: In scope SELECT today([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:17.960538 [ 2707 ] {3ea52300-8a53-4e8e-9a7b-8c195c59d439} executeQuery: Code: 42. DB::Exception: Number of arguments for function tokens doesn't match: passed 3, should be 1: In scope SELECT tokens([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tokens([], [], []);) 2025.09.25 21:12:17.965688 [ 2707 ] {3ea52300-8a53-4e8e-9a7b-8c195c59d439} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tokens doesn't match: passed 3, should be 1: In scope SELECT tokens([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:18.069573 [ 2707 ] {d1f5ed33-bf36-4813-aae1-474e3e1a6d26} executeQuery: Code: 42. DB::Exception: Aggregate function topK requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT topK([], [], []);) 2025.09.25 21:12:18.077678 [ 2707 ] {d1f5ed33-bf36-4813-aae1-474e3e1a6d26} TCPHandler: Code: 42. DB::Exception: Aggregate function topK requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:18.174285 [ 2707 ] {1ff115b8-fad4-436d-8cd6-2eca59fde454} executeQuery: Code: 42. DB::Exception: Aggregate function topKWeighted requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT topKWeighted([], [], []);) 2025.09.25 21:12:18.182388 [ 2707 ] {1ff115b8-fad4-436d-8cd6-2eca59fde454} TCPHandler: Code: 42. DB::Exception: Aggregate function topKWeighted requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:18.272880 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:18.316209 [ 2707 ] {ff6b4da4-8192-4dd7-a8fa-c5d9659373f6} executeQuery: Code: 42. DB::Exception: Number of arguments for function topLevelDomain doesn't match: passed 3, should be 1: In scope SELECT topLevelDomain([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT topLevelDomain([], [], []);) 2025.09.25 21:12:18.317542 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:18.321336 [ 2707 ] {ff6b4da4-8192-4dd7-a8fa-c5d9659373f6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function topLevelDomain doesn't match: passed 3, should be 1: In scope SELECT topLevelDomain([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:18.377520 [ 2707 ] {dc5ec4ad-5024-4a00-9f3f-830f2d6adcc8} executeQuery: Code: 42. DB::Exception: Number of arguments for function topLevelDomainRFC doesn't match: passed 3, should be 1: In scope SELECT topLevelDomainRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT topLevelDomainRFC([], [], []);) 2025.09.25 21:12:18.389540 [ 2707 ] {dc5ec4ad-5024-4a00-9f3f-830f2d6adcc8} TCPHandler: Code: 42. DB::Exception: Number of arguments for function topLevelDomainRFC doesn't match: passed 3, should be 1: In scope SELECT topLevelDomainRFC([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:18.525079 [ 2707 ] {9366e24c-ea21-4c04-a64d-1db71d96aa21} executeQuery: Code: 42. DB::Exception: Number of arguments for function transactionID doesn't match: passed 3, should be 0: In scope SELECT transactionID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT transactionID([], [], []);) 2025.09.25 21:12:18.530194 [ 2707 ] {9366e24c-ea21-4c04-a64d-1db71d96aa21} TCPHandler: Code: 42. DB::Exception: Number of arguments for function transactionID doesn't match: passed 3, should be 0: In scope SELECT transactionID([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:18.577226 [ 2707 ] {697dfa4c-c33e-47ef-a0fc-7816021ce67f} executeQuery: Code: 42. DB::Exception: Number of arguments for function transactionLatestSnapshot doesn't match: passed 3, should be 0: In scope SELECT transactionLatestSnapshot([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT transactionLatestSnapshot([], [], []);) 2025.09.25 21:12:18.583153 [ 2707 ] {697dfa4c-c33e-47ef-a0fc-7816021ce67f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function transactionLatestSnapshot doesn't match: passed 3, should be 0: In scope SELECT transactionLatestSnapshot([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:18.760662 [ 2707 ] {57568487-cb48-4e81-b8ba-6ec031656f11} executeQuery: Code: 42. DB::Exception: Number of arguments for function transactionOldestSnapshot doesn't match: passed 3, should be 0: In scope SELECT transactionOldestSnapshot([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT transactionOldestSnapshot([], [], []);) 2025.09.25 21:12:18.767676 [ 2707 ] {57568487-cb48-4e81-b8ba-6ec031656f11} TCPHandler: Code: 42. DB::Exception: Number of arguments for function transactionOldestSnapshot doesn't match: passed 3, should be 0: In scope SELECT transactionOldestSnapshot([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:18.838329 [ 2707 ] {402da91d-9cbc-474c-8540-fdc94a026618} executeQuery: Code: 36. DB::Exception: Unexpected type Array(Nothing) in function 'transform': In scope SELECT transform([], [], []). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT transform([], [], []);) 2025.09.25 21:12:18.843486 [ 2707 ] {402da91d-9cbc-474c-8540-fdc94a026618} TCPHandler: Code: 36. DB::Exception: Unexpected type Array(Nothing) in function 'transform': In scope SELECT transform([], [], []). (BAD_ARGUMENTS) 2025.09.25 21:12:18.892194 [ 2707 ] {92c8e243-7944-42d4-8155-711d5bd3793e} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function translate: In scope SELECT translate([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT translate([], [], []);) 2025.09.25 21:12:18.897693 [ 2707 ] {92c8e243-7944-42d4-8155-711d5bd3793e} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function translate: In scope SELECT translate([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:18.948186 [ 2707 ] {07c8c2a3-757b-48a5-b7f4-18fdd1c49a66} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function translateUTF8: In scope SELECT translateUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT translateUTF8([], [], []);) 2025.09.25 21:12:18.952346 [ 2707 ] {07c8c2a3-757b-48a5-b7f4-18fdd1c49a66} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function translateUTF8: In scope SELECT translateUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:18.995839 [ 2707 ] {bb5de20b-6b51-4ca5-a855-1039c1878c78} executeQuery: Code: 42. DB::Exception: Number of arguments for function trimBoth doesn't match: passed 3, should be 1: In scope SELECT trimBoth([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT trimBoth([], [], []);) 2025.09.25 21:12:19.000326 [ 2707 ] {bb5de20b-6b51-4ca5-a855-1039c1878c78} TCPHandler: Code: 42. DB::Exception: Number of arguments for function trimBoth doesn't match: passed 3, should be 1: In scope SELECT trimBoth([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:19.020494 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:19.039396 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:19.142076 [ 2707 ] {0e88c026-9cd6-4058-a97f-07025e4eda1e} executeQuery: Code: 42. DB::Exception: Number of arguments for function trimLeft doesn't match: passed 3, should be 1: In scope SELECT trimLeft([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT trimLeft([], [], []);) 2025.09.25 21:12:19.146334 [ 2707 ] {0e88c026-9cd6-4058-a97f-07025e4eda1e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function trimLeft doesn't match: passed 3, should be 1: In scope SELECT trimLeft([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:19.201806 [ 2707 ] {ef76b16b-d11e-42e8-9b30-2b673b066da2} executeQuery: Code: 42. DB::Exception: Number of arguments for function trimRight doesn't match: passed 3, should be 1: In scope SELECT trimRight([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT trimRight([], [], []);) 2025.09.25 21:12:19.207612 [ 2707 ] {ef76b16b-d11e-42e8-9b30-2b673b066da2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function trimRight doesn't match: passed 3, should be 1: In scope SELECT trimRight([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:19.259849 [ 2707 ] {4dc11969-c7c7-4f9d-a5b5-bd85f50c2b8b} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'trunc'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT trunc([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT trunc([], [], []);) 2025.09.25 21:12:19.267265 [ 2707 ] {4dc11969-c7c7-4f9d-a5b5-bd85f50c2b8b} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'trunc'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT trunc([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:19.322779 [ 7168 ] {ada14bb1-abb0-49b1-88a4-e7fc0524b5b6} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43970) (comment: 01034_JSONCompactEachRow.sql) (in query: SELECT * FROM test_table FORMAT JSONCompactEachRow;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:19.330807 [ 2707 ] {7588172b-7cf4-4524-a509-9a72da114b61} executeQuery: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'trunc'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT truncate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT truncate([], [], []);) 2025.09.25 21:12:19.335710 [ 2707 ] {7588172b-7cf4-4524-a509-9a72da114b61} TCPHandler: Code: 42. DB::Exception: An incorrect number of arguments was specified for function 'trunc'. Expected 1 mandatory argument and 1 optional argument, got 3 arguments: In scope SELECT truncate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:19.336383 [ 7168 ] {ada14bb1-abb0-49b1-88a4-e7fc0524b5b6} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:19.475970 [ 2707 ] {faeaf770-d625-491a-9f45-685a4b446729} executeQuery: Code: 42. DB::Exception: Number of arguments for function tryBase58Decode doesn't match: passed 3, should be 1: In scope SELECT tryBase58Decode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tryBase58Decode([], [], []);) 2025.09.25 21:12:19.484786 [ 2707 ] {faeaf770-d625-491a-9f45-685a4b446729} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tryBase58Decode doesn't match: passed 3, should be 1: In scope SELECT tryBase58Decode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:19.593860 [ 3619 ] {c94634a0-c520-4270-adf9-416dc5ae7ab1} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:36834) (comment: 01034_JSONCompactEachRow.sql) (in query: DROP DATABASE IF EXISTS test_gxf64rly ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:19.599392 [ 3619 ] {c94634a0-c520-4270-adf9-416dc5ae7ab1} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:19.619757 [ 2707 ] {01d058e9-6c0e-4743-97fc-531d3e67479d} executeQuery: Code: 42. DB::Exception: Number of arguments for function tryBase64Decode doesn't match: passed 3, should be 1: In scope SELECT tryBase64Decode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tryBase64Decode([], [], []);) 2025.09.25 21:12:19.625477 [ 2707 ] {01d058e9-6c0e-4743-97fc-531d3e67479d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tryBase64Decode doesn't match: passed 3, should be 1: In scope SELECT tryBase64Decode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:19.677948 [ 2707 ] {678f5a53-eea6-43bc-92e9-35a021e4e2e2} executeQuery: Code: 42. DB::Exception: Number of arguments for function tryBase64URLDecode doesn't match: passed 3, should be 1: In scope SELECT tryBase64URLDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tryBase64URLDecode([], [], []);) 2025.09.25 21:12:19.682080 [ 2707 ] {678f5a53-eea6-43bc-92e9-35a021e4e2e2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tryBase64URLDecode doesn't match: passed 3, should be 1: In scope SELECT tryBase64URLDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:19.689450 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:19.708398 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:19.731657 [ 2707 ] {35b2586e-8e09-4a5f-ae1b-2d7051c4bb1f} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'mode' to function 'tryDecrypt'. Expected: decryption mode string, got: Array(Nothing): In scope SELECT tryDecrypt([], '[HIDDEN]'). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tryDecrypt([], '[HIDDEN]')) 2025.09.25 21:12:19.736791 [ 2707 ] {35b2586e-8e09-4a5f-ae1b-2d7051c4bb1f} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'mode' to function 'tryDecrypt'. Expected: decryption mode string, got: Array(Nothing): In scope SELECT tryDecrypt([], '[HIDDEN]'). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:19.802869 [ 2707 ] {26bd76b5-f6c8-4b8e-9f88-ca4d7367872d} executeQuery: Code: 42. DB::Exception: Number of arguments for function tryIdnaEncode doesn't match: passed 3, should be 1: In scope SELECT tryIdnaEncode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tryIdnaEncode([], [], []);) 2025.09.25 21:12:19.808117 [ 2707 ] {26bd76b5-f6c8-4b8e-9f88-ca4d7367872d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tryIdnaEncode doesn't match: passed 3, should be 1: In scope SELECT tryIdnaEncode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:19.809039 [ 7168 ] {cb03f48f-7512-468a-a886-1e1d4bba0526} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:36848) (comment: 01034_JSONCompactEachRow.sql) (in query: DROP DATABASE IF EXISTS test_gxf64rly ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:19.813520 [ 7168 ] {cb03f48f-7512-468a-a886-1e1d4bba0526} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:19.868090 [ 2707 ] {e3b83b02-aee9-4643-872c-ca8609d9cd30} executeQuery: Code: 42. DB::Exception: Number of arguments for function tryPunycodeDecode doesn't match: passed 3, should be 1: In scope SELECT tryPunycodeDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tryPunycodeDecode([], [], []);) 2025.09.25 21:12:19.873692 [ 2707 ] {e3b83b02-aee9-4643-872c-ca8609d9cd30} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tryPunycodeDecode doesn't match: passed 3, should be 1: In scope SELECT tryPunycodeDecode([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:19.923815 [ 2707 ] {d671fa8b-5c86-4ba3-b003-461fa8e647d9} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function tumble. Should be a date with time: In scope SELECT tumble([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tumble([], [], []);) 2025.09.25 21:12:19.928777 [ 2707 ] {d671fa8b-5c86-4ba3-b003-461fa8e647d9} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function tumble. Should be a date with time: In scope SELECT tumble([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:19.952085 [ 3619 ] {962d2be8-95bb-4332-a49a-70f84d3fe649} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:36850) (comment: 01034_JSONCompactEachRow.sql) (in query: DROP DATABASE IF EXISTS test_gxf64rly ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000291f4e6a 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 13. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 14. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 15. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 19. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 20. asan_thread_start(void*) @ 0x000000000aa43059 21. ? @ 0x00007f12cd75eac3 22. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:19.955047 [ 3619 ] {962d2be8-95bb-4332-a49a-70f84d3fe649} DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000291f4e6a 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 13. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 14. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 15. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 19. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 20. asan_thread_start(void*) @ 0x000000000aa43059 21. ? @ 0x00007f12cd75eac3 22. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:19.983554 [ 2707 ] {a6b706b7-0740-41ec-9e2c-cedc53b61b97} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function tumbleEnd. Should be a date with time: In scope SELECT tumbleEnd([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tumbleEnd([], [], []);) 2025.09.25 21:12:19.988358 [ 2707 ] {a6b706b7-0740-41ec-9e2c-cedc53b61b97} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function tumbleEnd. Should be a date with time: In scope SELECT tumbleEnd([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:20.054235 [ 2707 ] {6ea489fd-3d3e-4861-827e-7ec729dcd4b4} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function tumbleStart. Should be a date with time: In scope SELECT tumbleStart([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tumbleStart([], [], []);) 2025.09.25 21:12:20.059969 [ 2707 ] {6ea489fd-3d3e-4861-827e-7ec729dcd4b4} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function tumbleStart. Should be a date with time: In scope SELECT tumbleStart([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:20.073694 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:20.186412 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:20.228083 [ 2707 ] {b6fa8f47-3949-451d-b40d-840ec38d6c70} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=725). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tuple([], [], []);) 2025.09.25 21:12:20.236183 [ 2707 ] {b6fa8f47-3949-451d-b40d-840ec38d6c70} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=725). (CANNOT_SCHEDULE_TASK) 2025.09.25 21:12:20.421697 [ 2707 ] {08df7976-058f-4aa9-b499-4ee609adee35} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument 1 of function tupleConcat: In scope SELECT tupleConcat([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleConcat([], [], []);) 2025.09.25 21:12:20.426315 [ 2707 ] {08df7976-058f-4aa9-b499-4ee609adee35} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument 1 of function tupleConcat: In scope SELECT tupleConcat([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:20.533673 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:20.567960 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:20.581738 [ 2707 ] {df53a7e6-1ac1-4d09-8d91-a100a98dffa5} executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleDivide doesn't match: passed 3, should be 2: In scope SELECT tupleDivide([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleDivide([], [], []);) 2025.09.25 21:12:20.590902 [ 2707 ] {df53a7e6-1ac1-4d09-8d91-a100a98dffa5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleDivide doesn't match: passed 3, should be 2: In scope SELECT tupleDivide([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:20.730796 [ 2707 ] {20115396-e323-43de-80d7-8b949f7e262f} executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleDivideByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleDivideByNumber([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleDivideByNumber([], [], []);) 2025.09.25 21:12:20.736522 [ 2707 ] {20115396-e323-43de-80d7-8b949f7e262f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleDivideByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleDivideByNumber([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:20.807060 [ 2707 ] {0e0854ba-f157-45b4-913c-040e03b5d507} executeQuery: Code: 43. DB::Exception: First argument for function tupleElement must be tuple or array of tuple. Actual Array(Nothing): In scope SELECT tupleElement([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleElement([], [], []);) 2025.09.25 21:12:20.821866 [ 2707 ] {0e0854ba-f157-45b4-913c-040e03b5d507} TCPHandler: Code: 43. DB::Exception: First argument for function tupleElement must be tuple or array of tuple. Actual Array(Nothing): In scope SELECT tupleElement([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:20.943069 [ 2707 ] {213188fe-5ec0-4045-98e5-88a868b12cd6} executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleHammingDistance doesn't match: passed 3, should be 2: In scope SELECT tupleHammingDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleHammingDistance([], [], []);) 2025.09.25 21:12:20.952967 [ 2707 ] {213188fe-5ec0-4045-98e5-88a868b12cd6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleHammingDistance doesn't match: passed 3, should be 2: In scope SELECT tupleHammingDistance([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:21.060538 [ 2707 ] {fc19b278-c0b8-4dd5-bea0-85c60f950cc5} executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleIntDiv doesn't match: passed 3, should be 2: In scope SELECT tupleIntDiv([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleIntDiv([], [], []);) 2025.09.25 21:12:21.068120 [ 2707 ] {fc19b278-c0b8-4dd5-bea0-85c60f950cc5} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleIntDiv doesn't match: passed 3, should be 2: In scope SELECT tupleIntDiv([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:21.183804 [ 2707 ] {cf7e440f-91a1-4f5f-a811-60b61d307888} executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleIntDivByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleIntDivByNumber([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleIntDivByNumber([], [], []);) 2025.09.25 21:12:21.198289 [ 2707 ] {cf7e440f-91a1-4f5f-a811-60b61d307888} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleIntDivByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleIntDivByNumber([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:21.202025 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:21.236152 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:21.289975 [ 2707 ] {738be207-1c1f-4428-a1fd-1d2d18386231} executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleIntDivOrZero doesn't match: passed 3, should be 2: In scope SELECT tupleIntDivOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleIntDivOrZero([], [], []);) 2025.09.25 21:12:21.344334 [ 2707 ] {738be207-1c1f-4428-a1fd-1d2d18386231} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleIntDivOrZero doesn't match: passed 3, should be 2: In scope SELECT tupleIntDivOrZero([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:21.451446 [ 2707 ] {d578b4b0-eebd-4458-b06a-efaf897ee65d} executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleIntDivOrZeroByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleIntDivOrZeroByNumber([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleIntDivOrZeroByNumber([], [], []);) 2025.09.25 21:12:21.461606 [ 2707 ] {d578b4b0-eebd-4458-b06a-efaf897ee65d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleIntDivOrZeroByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleIntDivOrZeroByNumber([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:21.531112 [ 2707 ] {90d7d36b-65dc-429c-bafd-7944dce0913c} executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleMinus doesn't match: passed 3, should be 2: In scope SELECT tupleMinus([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleMinus([], [], []);) 2025.09.25 21:12:21.538814 [ 2707 ] {90d7d36b-65dc-429c-bafd-7944dce0913c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleMinus doesn't match: passed 3, should be 2: In scope SELECT tupleMinus([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:21.730297 [ 2707 ] {23a168d8-0f8f-4beb-bb9b-cdc52ccfb7ec} executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleModulo doesn't match: passed 3, should be 2: In scope SELECT tupleModulo([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleModulo([], [], []);) 2025.09.25 21:12:21.733930 [ 2707 ] {23a168d8-0f8f-4beb-bb9b-cdc52ccfb7ec} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleModulo doesn't match: passed 3, should be 2: In scope SELECT tupleModulo([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:21.792835 [ 2707 ] {ccff3411-ba5c-4038-8674-7c0b1621580e} executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleModuloByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleModuloByNumber([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleModuloByNumber([], [], []);) 2025.09.25 21:12:21.803969 [ 2707 ] {ccff3411-ba5c-4038-8674-7c0b1621580e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleModuloByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleModuloByNumber([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:21.884278 [ 2707 ] {5abc1f5d-fd76-4b47-af2a-ffed0aa7afe3} executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleMultiply doesn't match: passed 3, should be 2: In scope SELECT tupleMultiply([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleMultiply([], [], []);) 2025.09.25 21:12:21.887470 [ 2707 ] {5abc1f5d-fd76-4b47-af2a-ffed0aa7afe3} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleMultiply doesn't match: passed 3, should be 2: In scope SELECT tupleMultiply([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:21.947412 [ 2707 ] {1500e5f3-bc45-4ec5-a3a0-3a5b6de4f6c4} executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleMultiplyByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleMultiplyByNumber([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleMultiplyByNumber([], [], []);) 2025.09.25 21:12:21.961039 [ 2707 ] {1500e5f3-bc45-4ec5-a3a0-3a5b6de4f6c4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleMultiplyByNumber doesn't match: passed 3, should be 2: In scope SELECT tupleMultiplyByNumber([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:22.057604 [ 2707 ] {58d4ac01-1922-4673-92d8-48d7e1e9e0a4} executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleNames doesn't match: passed 3, should be 1: In scope SELECT tupleNames([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleNames([], [], []);) 2025.09.25 21:12:22.061422 [ 2707 ] {58d4ac01-1922-4673-92d8-48d7e1e9e0a4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleNames doesn't match: passed 3, should be 1: In scope SELECT tupleNames([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:22.226531 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:22.236106 [ 2707 ] {084bf77e-a806-43b3-86ac-883fc069461d} executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleNegate doesn't match: passed 3, should be 1: In scope SELECT tupleNegate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleNegate([], [], []);) 2025.09.25 21:12:22.239769 [ 2707 ] {084bf77e-a806-43b3-86ac-883fc069461d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleNegate doesn't match: passed 3, should be 1: In scope SELECT tupleNegate([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:22.281978 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:22.335292 [ 2707 ] {ffcb03bd-c1de-4d95-8bf1-fa673bf341c2} executeQuery: Code: 42. DB::Exception: Number of arguments for function tuplePlus doesn't match: passed 3, should be 2: In scope SELECT tuplePlus([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tuplePlus([], [], []);) 2025.09.25 21:12:22.405450 [ 2707 ] {ffcb03bd-c1de-4d95-8bf1-fa673bf341c2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tuplePlus doesn't match: passed 3, should be 2: In scope SELECT tuplePlus([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:22.520945 [ 2707 ] {b1cc2f41-92a8-4e49-99b7-9c454baac393} executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleToNameValuePairs doesn't match: passed 3, should be 1: In scope SELECT tupleToNameValuePairs([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT tupleToNameValuePairs([], [], []);) 2025.09.25 21:12:22.527099 [ 2707 ] {b1cc2f41-92a8-4e49-99b7-9c454baac393} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleToNameValuePairs doesn't match: passed 3, should be 1: In scope SELECT tupleToNameValuePairs([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:22.609450 [ 2707 ] {9cf929bb-0726-43f7-9181-62f1986c7a8f} executeQuery: Code: 42. DB::Exception: Number of arguments for function upper doesn't match: passed 3, should be 1: In scope SELECT ucase([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT ucase([], [], []);) 2025.09.25 21:12:22.616881 [ 2707 ] {9cf929bb-0726-43f7-9181-62f1986c7a8f} TCPHandler: Code: 42. DB::Exception: Number of arguments for function upper doesn't match: passed 3, should be 1: In scope SELECT ucase([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:22.667207 [ 2707 ] {b2261578-2b26-4663-a78f-27e0f25abe5c} executeQuery: Code: 42. DB::Exception: Number of arguments for function unbin doesn't match: passed 3, should be 1: In scope SELECT unbin([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT unbin([], [], []);) 2025.09.25 21:12:22.670563 [ 2707 ] {b2261578-2b26-4663-a78f-27e0f25abe5c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function unbin doesn't match: passed 3, should be 1: In scope SELECT unbin([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:22.714783 [ 2707 ] {6acad703-c9f1-40ca-b8c3-56a765d58c7a} executeQuery: Code: 42. DB::Exception: Number of arguments for function unhex doesn't match: passed 3, should be 1: In scope SELECT unhex([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT unhex([], [], []);) 2025.09.25 21:12:22.718654 [ 2707 ] {6acad703-c9f1-40ca-b8c3-56a765d58c7a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function unhex doesn't match: passed 3, should be 1: In scope SELECT unhex([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:23.368088 [ 2707 ] {4551f145-c1f6-40f6-87dd-5c0b1538e0e6} executeQuery: Code: 42. DB::Exception: Number of arguments for function uniqThetaIntersect doesn't match: passed 3, should be 2: In scope SELECT uniqThetaIntersect([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT uniqThetaIntersect([], [], []);) 2025.09.25 21:12:23.371593 [ 2707 ] {4551f145-c1f6-40f6-87dd-5c0b1538e0e6} TCPHandler: Code: 42. DB::Exception: Number of arguments for function uniqThetaIntersect doesn't match: passed 3, should be 2: In scope SELECT uniqThetaIntersect([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:23.427616 [ 2707 ] {b504ad2b-3179-4c38-ab9d-c5f00a232134} executeQuery: Code: 42. DB::Exception: Number of arguments for function uniqThetaNot doesn't match: passed 3, should be 2: In scope SELECT uniqThetaNot([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT uniqThetaNot([], [], []);) 2025.09.25 21:12:23.435713 [ 2707 ] {b504ad2b-3179-4c38-ab9d-c5f00a232134} TCPHandler: Code: 42. DB::Exception: Number of arguments for function uniqThetaNot doesn't match: passed 3, should be 2: In scope SELECT uniqThetaNot([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:23.437449 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:23.448685 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:23.499203 [ 2707 ] {17103c9e-44f8-455c-a1d2-d5ece860bb58} executeQuery: Code: 42. DB::Exception: Number of arguments for function uniqThetaUnion doesn't match: passed 3, should be 2: In scope SELECT uniqThetaUnion([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT uniqThetaUnion([], [], []);) 2025.09.25 21:12:23.507696 [ 2707 ] {17103c9e-44f8-455c-a1d2-d5ece860bb58} TCPHandler: Code: 42. DB::Exception: Number of arguments for function uniqThetaUnion doesn't match: passed 3, should be 2: In scope SELECT uniqThetaUnion([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:23.665839 [ 2707 ] {9cc5a263-9671-487b-84cd-c6910d046471} executeQuery: Code: 42. DB::Exception: Number of arguments for function upper doesn't match: passed 3, should be 1: In scope SELECT upper([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT upper([], [], []);) 2025.09.25 21:12:23.668902 [ 2707 ] {9cc5a263-9671-487b-84cd-c6910d046471} TCPHandler: Code: 42. DB::Exception: Number of arguments for function upper doesn't match: passed 3, should be 1: In scope SELECT upper([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:23.699133 [ 2707 ] {dec3c98f-d8f7-4af3-9a61-e2fd1d8545be} executeQuery: Code: 42. DB::Exception: Number of arguments for function upperUTF8 doesn't match: passed 3, should be 1: In scope SELECT upperUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT upperUTF8([], [], []);) 2025.09.25 21:12:23.702557 [ 2707 ] {dec3c98f-d8f7-4af3-9a61-e2fd1d8545be} TCPHandler: Code: 42. DB::Exception: Number of arguments for function upperUTF8 doesn't match: passed 3, should be 1: In scope SELECT upperUTF8([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:23.712297 [ 8169 ] {5e3f5660-a1da-4f70-a3c9-df319000ef3e} AsynchronousInsertQueue: Failed parsing for query 'INSERT INTO test_nay98swp.t_async_inserts_logs FORMAT JSONEachRow' with query id 12815184-b5ba-41d1-b9ce-91f95a0c97ce. DB::Exception: Cannot parse input: expected '{' before: 'qqqqqq': (at row 1) 2025.09.25 21:12:23.745820 [ 3436 ] {7836c69d-e718-4005-bddf-518e9ab2099d} AsynchronousInsertQueue: Failed parsing for query 'INSERT INTO test_nay98swp.t_async_inserts_logs FORMAT Values' with query id bf63e226-0e8e-4f02-bfc4-c2bb27220f03. DB::Exception: Cannot parse input: expected '(' before: 'qqqqqq': at row 0 2025.09.25 21:12:23.748480 [ 7168 ] {12815184-b5ba-41d1-b9ce-91f95a0c97ce} executeQuery: Code: 27. DB::Exception: Cannot parse input: expected '{' before: 'qqqqqq': (at row 1) : While executing WaitForAsyncInsert. (CANNOT_PARSE_INPUT_ASSERTION_FAILED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:55864) (comment: 02456_async_inserts_logs.sh) (in query: INSERT INTO t_async_inserts_logs FORMAT JSONEachRow ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::throwAtAssertionFailed(char const*, DB::ReadBuffer&) @ 0x000000001aeb2ed5 4. ./src/IO/ReadBuffer.h:114: DB::JSONEachRowRowInputFormat::readJSONObject(std::vector::mutable_ptr, std::allocator::mutable_ptr>>&) @ 0x000000002d620bd9 5. ./src/Processors/ISource.h:41: DB::JSONEachRowRowInputFormat::readRow(std::vector::mutable_ptr, std::allocator::mutable_ptr>>&, DB::RowReadExtension&) @ 0x000000002d6214a7 6. ./build_docker/./src/Processors/Formats/IRowInputFormat.cpp:0: DB::IRowInputFormat::read() @ 0x000000002d46d965 7. ./build_docker/./src/Processors/Formats/IInputFormat.cpp:19: DB::IInputFormat::generate() @ 0x000000002d37a746 8. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 9. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 10. ./build_docker/./src/Processors/Executors/StreamingFormatExecutor.cpp:59: DB::StreamingFormatExecutor::execute() @ 0x000000002d377dad 11. ./build_docker/./src/Processors/Executors/StreamingFormatExecutor.cpp:44: DB::StreamingFormatExecutor::execute(DB::ReadBuffer&) @ 0x000000002d37775e 12. ./build_docker/./src/Interpreters/AsynchronousInsertQueue.cpp:0: DB::AsynchronousInsertQueue::processData(DB::AsynchronousInsertQueue::InsertQuery, std::unique_ptr>, std::shared_ptr, DB::AsynchronousInsertQueue::QueueShardFlushTimeHistory&) @ 0x0000000028490249 13. ./build_docker/./src/Interpreters/AsynchronousInsertQueue.cpp:289: void std::__function::__policy_invoker::__call_impl>, std::shared_ptr, unsigned long)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000028497489 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:23.760542 [ 7168 ] {12815184-b5ba-41d1-b9ce-91f95a0c97ce} DynamicQueryHandler: Code: 27. DB::Exception: Cannot parse input: expected '{' before: 'qqqqqq': (at row 1) : While executing WaitForAsyncInsert. (CANNOT_PARSE_INPUT_ASSERTION_FAILED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::throwAtAssertionFailed(char const*, DB::ReadBuffer&) @ 0x000000001aeb2ed5 4. ./src/IO/ReadBuffer.h:114: DB::JSONEachRowRowInputFormat::readJSONObject(std::vector::mutable_ptr, std::allocator::mutable_ptr>>&) @ 0x000000002d620bd9 5. ./src/Processors/ISource.h:41: DB::JSONEachRowRowInputFormat::readRow(std::vector::mutable_ptr, std::allocator::mutable_ptr>>&, DB::RowReadExtension&) @ 0x000000002d6214a7 6. ./build_docker/./src/Processors/Formats/IRowInputFormat.cpp:0: DB::IRowInputFormat::read() @ 0x000000002d46d965 7. ./build_docker/./src/Processors/Formats/IInputFormat.cpp:19: DB::IInputFormat::generate() @ 0x000000002d37a746 8. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 9. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 10. ./build_docker/./src/Processors/Executors/StreamingFormatExecutor.cpp:59: DB::StreamingFormatExecutor::execute() @ 0x000000002d377dad 11. ./build_docker/./src/Processors/Executors/StreamingFormatExecutor.cpp:44: DB::StreamingFormatExecutor::execute(DB::ReadBuffer&) @ 0x000000002d37775e 12. ./build_docker/./src/Interpreters/AsynchronousInsertQueue.cpp:0: DB::AsynchronousInsertQueue::processData(DB::AsynchronousInsertQueue::InsertQuery, std::unique_ptr>, std::shared_ptr, DB::AsynchronousInsertQueue::QueueShardFlushTimeHistory&) @ 0x0000000028490249 13. ./build_docker/./src/Interpreters/AsynchronousInsertQueue.cpp:289: void std::__function::__policy_invoker::__call_impl>, std::shared_ptr, unsigned long)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000028497489 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:23.767794 [ 2707 ] {ae29b681-a1a0-4cef-b875-37592188037e} executeQuery: Code: 42. DB::Exception: Number of arguments for function uptime doesn't match: passed 3, should be 0: In scope SELECT uptime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT uptime([], [], []);) 2025.09.25 21:12:23.771098 [ 2707 ] {ae29b681-a1a0-4cef-b875-37592188037e} TCPHandler: Code: 42. DB::Exception: Number of arguments for function uptime doesn't match: passed 3, should be 0: In scope SELECT uptime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:23.824909 [ 3851 ] {bf63e226-0e8e-4f02-bfc4-c2bb27220f03} executeQuery: Code: 27. DB::Exception: Cannot parse input: expected '(' before: 'qqqqqq': at row 0: While executing WaitForAsyncInsert. (CANNOT_PARSE_INPUT_ASSERTION_FAILED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:55882) (comment: 02456_async_inserts_logs.sh) (in query: INSERT INTO t_async_inserts_logs VALUES ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::throwAtAssertionFailed(char const*, DB::ReadBuffer&) @ 0x000000001aeb2ed5 4. ./src/IO/ReadHelpers.h:1704: DB::ValuesBlockInputFormat::readRow(std::vector::mutable_ptr, std::allocator::mutable_ptr>>&, unsigned long) @ 0x000000002d9022b3 5. ./build_docker/./src/Processors/Formats/Impl/ValuesBlockInputFormat.cpp:0: DB::ValuesBlockInputFormat::read() @ 0x000000002d8ffd31 6. ./build_docker/./src/Processors/Formats/IInputFormat.cpp:19: DB::IInputFormat::generate() @ 0x000000002d37a746 7. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 8. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 9. ./build_docker/./src/Processors/Executors/StreamingFormatExecutor.cpp:59: DB::StreamingFormatExecutor::execute() @ 0x000000002d377dad 10. ./build_docker/./src/Processors/Executors/StreamingFormatExecutor.cpp:44: DB::StreamingFormatExecutor::execute(DB::ReadBuffer&) @ 0x000000002d37775e 11. ./build_docker/./src/Interpreters/AsynchronousInsertQueue.cpp:0: DB::AsynchronousInsertQueue::processData(DB::AsynchronousInsertQueue::InsertQuery, std::unique_ptr>, std::shared_ptr, DB::AsynchronousInsertQueue::QueueShardFlushTimeHistory&) @ 0x0000000028490249 12. ./build_docker/./src/Interpreters/AsynchronousInsertQueue.cpp:289: void std::__function::__policy_invoker::__call_impl>, std::shared_ptr, unsigned long)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000028497489 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 14. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 15. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:23.827697 [ 3851 ] {bf63e226-0e8e-4f02-bfc4-c2bb27220f03} DynamicQueryHandler: Code: 27. DB::Exception: Cannot parse input: expected '(' before: 'qqqqqq': at row 0: While executing WaitForAsyncInsert. (CANNOT_PARSE_INPUT_ASSERTION_FAILED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::throwAtAssertionFailed(char const*, DB::ReadBuffer&) @ 0x000000001aeb2ed5 4. ./src/IO/ReadHelpers.h:1704: DB::ValuesBlockInputFormat::readRow(std::vector::mutable_ptr, std::allocator::mutable_ptr>>&, unsigned long) @ 0x000000002d9022b3 5. ./build_docker/./src/Processors/Formats/Impl/ValuesBlockInputFormat.cpp:0: DB::ValuesBlockInputFormat::read() @ 0x000000002d8ffd31 6. ./build_docker/./src/Processors/Formats/IInputFormat.cpp:19: DB::IInputFormat::generate() @ 0x000000002d37a746 7. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 8. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 9. ./build_docker/./src/Processors/Executors/StreamingFormatExecutor.cpp:59: DB::StreamingFormatExecutor::execute() @ 0x000000002d377dad 10. ./build_docker/./src/Processors/Executors/StreamingFormatExecutor.cpp:44: DB::StreamingFormatExecutor::execute(DB::ReadBuffer&) @ 0x000000002d37775e 11. ./build_docker/./src/Interpreters/AsynchronousInsertQueue.cpp:0: DB::AsynchronousInsertQueue::processData(DB::AsynchronousInsertQueue::InsertQuery, std::unique_ptr>, std::shared_ptr, DB::AsynchronousInsertQueue::QueueShardFlushTimeHistory&) @ 0x0000000028490249 12. ./build_docker/./src/Interpreters/AsynchronousInsertQueue.cpp:289: void std::__function::__policy_invoker::__call_impl>, std::shared_ptr, unsigned long)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000028497489 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 14. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 15. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:23.881541 [ 2707 ] {042e87b6-2736-420d-a9ef-6fd00007d7a4} executeQuery: Code: 42. DB::Exception: Number of arguments for function currentUser doesn't match: passed 3, should be 0: In scope SELECT user([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT user([], [], []);) 2025.09.25 21:12:23.886738 [ 2707 ] {042e87b6-2736-420d-a9ef-6fd00007d7a4} TCPHandler: Code: 42. DB::Exception: Number of arguments for function currentUser doesn't match: passed 3, should be 0: In scope SELECT user([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:23.933114 [ 2707 ] {522a7f47-5aeb-4837-8992-fe2b1fc9bca9} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function validateNestedArraySizes Must be UInt.: In scope SELECT validateNestedArraySizes([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT validateNestedArraySizes([], [], []);) 2025.09.25 21:12:23.939847 [ 2707 ] {522a7f47-5aeb-4837-8992-fe2b1fc9bca9} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of first argument of function validateNestedArraySizes Must be UInt.: In scope SELECT validateNestedArraySizes([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:23.992123 [ 2707 ] {e7dfb23a-5d55-405a-8475-4cb38427d345} executeQuery: Code: 42. DB::Exception: Aggregate function varPop requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT varPop([], [], []);) 2025.09.25 21:12:24.003105 [ 2707 ] {e7dfb23a-5d55-405a-8475-4cb38427d345} TCPHandler: Code: 42. DB::Exception: Aggregate function varPop requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:24.171890 [ 2707 ] {7d772cfd-0ca5-4707-9da5-9cb53fcffa43} executeQuery: Code: 42. DB::Exception: Aggregate function varPopStable requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT varPopStable([], [], []);) 2025.09.25 21:12:24.174250 [ 2707 ] {7d772cfd-0ca5-4707-9da5-9cb53fcffa43} TCPHandler: Code: 42. DB::Exception: Aggregate function varPopStable requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:24.245040 [ 2707 ] {6f1a3141-e48a-4a19-bbdd-6e466f498eb1} executeQuery: Code: 42. DB::Exception: Aggregate function varSamp requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT varSamp([], [], []);) 2025.09.25 21:12:24.248879 [ 2707 ] {6f1a3141-e48a-4a19-bbdd-6e466f498eb1} TCPHandler: Code: 42. DB::Exception: Aggregate function varSamp requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:24.301001 [ 2707 ] {99c3922a-cd20-4dc9-96b1-65dff8f069b8} executeQuery: Code: 42. DB::Exception: Aggregate function varSampStable requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT varSampStable([], [], []);) 2025.09.25 21:12:24.329287 [ 2707 ] {99c3922a-cd20-4dc9-96b1-65dff8f069b8} TCPHandler: Code: 42. DB::Exception: Aggregate function varSampStable requires single argument. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:24.448450 [ 2707 ] {f3b271c8-2840-4f09-93f7-281866e2e5e9} executeQuery: Code: 43. DB::Exception: First argument for function variantElement must be Variant or Array of Variant. Actual Array(Nothing): In scope SELECT variantElement([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT variantElement([], [], []);) 2025.09.25 21:12:24.451301 [ 2707 ] {f3b271c8-2840-4f09-93f7-281866e2e5e9} TCPHandler: Code: 43. DB::Exception: First argument for function variantElement must be Variant or Array of Variant. Actual Array(Nothing): In scope SELECT variantElement([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:24.476473 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:24.504148 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:24.518384 [ 2707 ] {be527495-4151-4903-9c18-33e3d9c04593} executeQuery: Code: 42. DB::Exception: Number of arguments for function variantType doesn't match: passed 3, should be 1: In scope SELECT variantType([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT variantType([], [], []);) 2025.09.25 21:12:24.524294 [ 2707 ] {be527495-4151-4903-9c18-33e3d9c04593} TCPHandler: Code: 42. DB::Exception: Number of arguments for function variantType doesn't match: passed 3, should be 1: In scope SELECT variantType([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:24.613669 [ 2707 ] {b3b8c9d1-39d1-49b4-b951-ebf77c93acc2} executeQuery: Code: 42. DB::Exception: Number of arguments for function tupleMinus doesn't match: passed 3, should be 2: In scope SELECT vectorDifference([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT vectorDifference([], [], []);) 2025.09.25 21:12:24.632638 [ 2707 ] {b3b8c9d1-39d1-49b4-b951-ebf77c93acc2} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tupleMinus doesn't match: passed 3, should be 2: In scope SELECT vectorDifference([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:24.704752 [ 2707 ] {e6548652-8477-4515-b338-8b1e0f2974ed} executeQuery: Code: 42. DB::Exception: Number of arguments for function tuplePlus doesn't match: passed 3, should be 2: In scope SELECT vectorSum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT vectorSum([], [], []);) 2025.09.25 21:12:24.711915 [ 2707 ] {e6548652-8477-4515-b338-8b1e0f2974ed} TCPHandler: Code: 42. DB::Exception: Number of arguments for function tuplePlus doesn't match: passed 3, should be 2: In scope SELECT vectorSum([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:24.786415 [ 2707 ] {c047ae8a-ab84-4cda-a4c8-e9229957c225} executeQuery: Code: 42. DB::Exception: Number of arguments for function version doesn't match: passed 3, should be 0: In scope SELECT version([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT version([], [], []);) 2025.09.25 21:12:24.797286 [ 2707 ] {c047ae8a-ab84-4cda-a4c8-e9229957c225} TCPHandler: Code: 42. DB::Exception: Number of arguments for function version doesn't match: passed 3, should be 0: In scope SELECT version([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:24.910460 [ 2707 ] {54a258d8-35fe-42c1-9206-4fc98cf299a9} executeQuery: Code: 42. DB::Exception: Number of arguments for function visibleWidth doesn't match: passed 3, should be 1: In scope SELECT visibleWidth([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT visibleWidth([], [], []);) 2025.09.25 21:12:24.916133 [ 2707 ] {54a258d8-35fe-42c1-9206-4fc98cf299a9} TCPHandler: Code: 42. DB::Exception: Number of arguments for function visibleWidth doesn't match: passed 3, should be 1: In scope SELECT visibleWidth([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:24.966892 [ 2707 ] {b579140a-bfee-4f8b-ac6a-8a87959aca2d} executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractBool doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractBool([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT visitParamExtractBool([], [], []);) 2025.09.25 21:12:24.971729 [ 2707 ] {b579140a-bfee-4f8b-ac6a-8a87959aca2d} TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractBool doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractBool([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:25.047084 [ 2707 ] {f3edd8d0-6758-4b02-acbd-b69935c7e559} executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractFloat doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractFloat([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT visitParamExtractFloat([], [], []);) 2025.09.25 21:12:25.072063 [ 2707 ] {f3edd8d0-6758-4b02-acbd-b69935c7e559} TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractFloat doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractFloat([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:25.115039 [ 2707 ] {66c7d0ae-b6d5-4fa0-8889-98510bb4aca0} executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractInt doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractInt([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT visitParamExtractInt([], [], []);) 2025.09.25 21:12:25.118919 [ 2707 ] {66c7d0ae-b6d5-4fa0-8889-98510bb4aca0} TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractInt doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractInt([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:25.154201 [ 2707 ] {eeb62426-1c7f-4dc9-8aa4-7168e6477cbc} executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractRaw doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractRaw([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT visitParamExtractRaw([], [], []);) 2025.09.25 21:12:25.157668 [ 2707 ] {eeb62426-1c7f-4dc9-8aa4-7168e6477cbc} TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractRaw doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractRaw([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:25.188341 [ 2707 ] {b5997890-c859-4a40-b646-c9102b85e730} executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractString doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT visitParamExtractString([], [], []);) 2025.09.25 21:12:25.192763 [ 2707 ] {b5997890-c859-4a40-b646-c9102b85e730} TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractString doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractString([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:25.231615 [ 2707 ] {f0d526eb-7485-4951-b866-fc2d6b352ed9} executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractUInt doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractUInt([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT visitParamExtractUInt([], [], []);) 2025.09.25 21:12:25.238344 [ 2707 ] {f0d526eb-7485-4951-b866-fc2d6b352ed9} TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONExtractUInt doesn't match: passed 3, should be 2: In scope SELECT visitParamExtractUInt([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:25.301331 [ 2707 ] {d1e69f10-1b4a-4043-8eea-b51f42ccf35c} executeQuery: Code: 42. DB::Exception: Number of arguments for function simpleJSONHas doesn't match: passed 3, should be 2: In scope SELECT visitParamHas([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT visitParamHas([], [], []);) 2025.09.25 21:12:25.308023 [ 2707 ] {d1e69f10-1b4a-4043-8eea-b51f42ccf35c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function simpleJSONHas doesn't match: passed 3, should be 2: In scope SELECT visitParamHas([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:25.313235 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:25.351354 [ 3587 ] {980c3216-a110-4471-8de5-a2f8ea2db1fb} executeQuery: Code: 252. DB::Exception: Too many parts (1 with average size of 555.00 B) in table 'test_nay98swp.t_async_inserts_logs (c7acec78-0fb5-40bb-bafa-06c1e757ef37)'. Merges are processing significantly slower than inserts. (TOO_MANY_PARTS) (version 24.8.14.10508.altinitytest (altinity build)) (from 0.0.0.0:0) (comment: 02456_async_inserts_logs.sh) (in query: INSERT INTO test_nay98swp.t_async_inserts_logs FORMAT Values), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, unsigned long&, ReadableSize&&, String&&) @ 0x000000002c558fa0 4. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:4585: DB::MergeTreeData::delayInsertOrThrowIfNeeded(Poco::Event*, std::shared_ptr const&, bool) const @ 0x000000002c4c5bec 5. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function, DB::ThreadStatus*, std::atomic*) @ 0x000000002d9c1551 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d9c04cf 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 11. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:113: DB::CompletedPipelineExecutor::execute() @ 0x000000002d34579e 12. ./build_docker/./src/Interpreters/AsynchronousInsertQueue.cpp:0: DB::AsynchronousInsertQueue::processData(DB::AsynchronousInsertQueue::InsertQuery, std::unique_ptr>, std::shared_ptr, DB::AsynchronousInsertQueue::QueueShardFlushTimeHistory&) @ 0x0000000028491faf 13. ./build_docker/./src/Interpreters/AsynchronousInsertQueue.cpp:289: void std::__function::__policy_invoker::__call_impl>, std::shared_ptr, unsigned long)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000028497489 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:25.362231 [ 2707 ] {ccf62421-8e92-4ca2-9ba3-16ec5503c081} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function toWeek. Must be Date, Date32, DateTime or DateTime64: In scope SELECT week([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT week([], [], []);) 2025.09.25 21:12:25.365050 [ 2707 ] {ccf62421-8e92-4ca2-9ba3-16ec5503c081} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function toWeek. Must be Date, Date32, DateTime or DateTime64: In scope SELECT week([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:25.404975 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:25.439858 [ 3587 ] {} AsynchronousInsertQueue: Failed insertion for query 'INSERT INTO test_nay98swp.t_async_inserts_logs FORMAT Values': Code: 252. DB::Exception: Too many parts (1 with average size of 555.00 B) in table 'test_nay98swp.t_async_inserts_logs (c7acec78-0fb5-40bb-bafa-06c1e757ef37)'. Merges are processing significantly slower than inserts. (TOO_MANY_PARTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, unsigned long&, ReadableSize&&, String&&) @ 0x000000002c558fa0 4. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:4585: DB::MergeTreeData::delayInsertOrThrowIfNeeded(Poco::Event*, std::shared_ptr const&, bool) const @ 0x000000002c4c5bec 5. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function, DB::ThreadStatus*, std::atomic*) @ 0x000000002d9c1551 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d9c04cf 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 11. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:113: DB::CompletedPipelineExecutor::execute() @ 0x000000002d34579e 12. ./build_docker/./src/Interpreters/AsynchronousInsertQueue.cpp:0: DB::AsynchronousInsertQueue::processData(DB::AsynchronousInsertQueue::InsertQuery, std::unique_ptr>, std::shared_ptr, DB::AsynchronousInsertQueue::QueueShardFlushTimeHistory&) @ 0x0000000028491faf 13. ./build_docker/./src/Interpreters/AsynchronousInsertQueue.cpp:289: void std::__function::__policy_invoker::__call_impl>, std::shared_ptr, unsigned long)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000028497489 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:25.443944 [ 3619 ] {0179bb28-c6e8-403a-ab8a-2b06ab5dc8ec} executeQuery: Code: 252. DB::Exception: Too many parts (1 with average size of 555.00 B) in table 'test_nay98swp.t_async_inserts_logs (c7acec78-0fb5-40bb-bafa-06c1e757ef37)'. Merges are processing significantly slower than inserts: While executing WaitForAsyncInsert. (TOO_MANY_PARTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:55910) (comment: 02456_async_inserts_logs.sh) (in query: INSERT INTO t_async_inserts_logs VALUES ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, unsigned long&, ReadableSize&&, String&&) @ 0x000000002c558fa0 4. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:4585: DB::MergeTreeData::delayInsertOrThrowIfNeeded(Poco::Event*, std::shared_ptr const&, bool) const @ 0x000000002c4c5bec 5. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function, DB::ThreadStatus*, std::atomic*) @ 0x000000002d9c1551 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d9c04cf 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 11. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:113: DB::CompletedPipelineExecutor::execute() @ 0x000000002d34579e 12. ./build_docker/./src/Interpreters/AsynchronousInsertQueue.cpp:0: DB::AsynchronousInsertQueue::processData(DB::AsynchronousInsertQueue::InsertQuery, std::unique_ptr>, std::shared_ptr, DB::AsynchronousInsertQueue::QueueShardFlushTimeHistory&) @ 0x0000000028491faf 13. ./build_docker/./src/Interpreters/AsynchronousInsertQueue.cpp:289: void std::__function::__policy_invoker::__call_impl>, std::shared_ptr, unsigned long)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000028497489 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:25.472990 [ 3619 ] {0179bb28-c6e8-403a-ab8a-2b06ab5dc8ec} DynamicQueryHandler: Code: 252. DB::Exception: Too many parts (1 with average size of 555.00 B) in table 'test_nay98swp.t_async_inserts_logs (c7acec78-0fb5-40bb-bafa-06c1e757ef37)'. Merges are processing significantly slower than inserts: While executing WaitForAsyncInsert. (TOO_MANY_PARTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, unsigned long&, ReadableSize&&, String&&) @ 0x000000002c558fa0 4. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:4585: DB::MergeTreeData::delayInsertOrThrowIfNeeded(Poco::Event*, std::shared_ptr const&, bool) const @ 0x000000002c4c5bec 5. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function, DB::ThreadStatus*, std::atomic*) @ 0x000000002d9c1551 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d9c04cf 7. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 11. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:113: DB::CompletedPipelineExecutor::execute() @ 0x000000002d34579e 12. ./build_docker/./src/Interpreters/AsynchronousInsertQueue.cpp:0: DB::AsynchronousInsertQueue::processData(DB::AsynchronousInsertQueue::InsertQuery, std::unique_ptr>, std::shared_ptr, DB::AsynchronousInsertQueue::QueueShardFlushTimeHistory&) @ 0x0000000028491faf 13. ./build_docker/./src/Interpreters/AsynchronousInsertQueue.cpp:289: void std::__function::__policy_invoker::__call_impl>, std::shared_ptr, unsigned long)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x0000000028497489 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:25.482746 [ 2707 ] {1f4a06a4-1f49-4865-ba23-968cb89dfce8} executeQuery: Code: 42. DB::Exception: Aggregate function welchTTest requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT welchTTest([], [], []);) 2025.09.25 21:12:25.494573 [ 2707 ] {1f4a06a4-1f49-4865-ba23-968cb89dfce8} TCPHandler: Code: 42. DB::Exception: Aggregate function welchTTest requires two arguments. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:25.586178 [ 2707 ] {1b9f556e-4beb-4301-bedc-040a2f285e0c} executeQuery: Code: 42. DB::Exception: Number of arguments for function widthBucket doesn't match: passed 3, should be 4: In scope SELECT widthBucket([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT widthBucket([], [], []);) 2025.09.25 21:12:25.590754 [ 2707 ] {1b9f556e-4beb-4301-bedc-040a2f285e0c} TCPHandler: Code: 42. DB::Exception: Number of arguments for function widthBucket doesn't match: passed 3, should be 4: In scope SELECT widthBucket([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:25.665183 [ 2707 ] {3c95eed5-a897-4c17-ae09-37e39c5c050a} executeQuery: Code: 42. DB::Exception: Number of arguments for function widthBucket doesn't match: passed 3, should be 4: In scope SELECT width_bucket([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT width_bucket([], [], []);) 2025.09.25 21:12:25.665398 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:25.674838 [ 2707 ] {3c95eed5-a897-4c17-ae09-37e39c5c050a} TCPHandler: Code: 42. DB::Exception: Number of arguments for function widthBucket doesn't match: passed 3, should be 4: In scope SELECT width_bucket([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:25.685818 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:25.749677 [ 2707 ] {855071db-6571-4ebe-866a-1df0b8780b3d} executeQuery: Code: 42. DB::Exception: Aggregate function windowFunnel requires at least one parameter: , [option, [option, ...]]. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT windowFunnel([], [], []);) 2025.09.25 21:12:25.757308 [ 2707 ] {855071db-6571-4ebe-866a-1df0b8780b3d} TCPHandler: Code: 42. DB::Exception: Aggregate function windowFunnel requires at least one parameter: , [option, [option, ...]]. (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:25.827680 [ 2707 ] {60c8e336-08c8-45ca-8a97-a180cbcde177} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function windowID. Should be a date with time: In scope SELECT windowID([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT windowID([], [], []);) 2025.09.25 21:12:25.884490 [ 2707 ] {60c8e336-08c8-45ca-8a97-a180cbcde177} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function windowID. Should be a date with time: In scope SELECT windowID([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:25.923425 [ 2707 ] {d460bb1f-ba98-41c4-9a6b-4aebb81ec434} executeQuery: Code: 42. DB::Exception: Number of arguments for function wkt doesn't match: passed 3, should be 1: In scope SELECT wkt([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT wkt([], [], []);) 2025.09.25 21:12:25.929158 [ 2707 ] {d460bb1f-ba98-41c4-9a6b-4aebb81ec434} TCPHandler: Code: 42. DB::Exception: Number of arguments for function wkt doesn't match: passed 3, should be 1: In scope SELECT wkt([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:26.063830 [ 2707 ] {68713a39-8746-43b1-83cf-733293873d04} executeQuery: Code: 43. DB::Exception: First argument of function wordShingleMinHash must be String, got Array(Nothing): In scope SELECT wordShingleMinHash([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT wordShingleMinHash([], [], []);) 2025.09.25 21:12:26.068698 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:26.076387 [ 2707 ] {68713a39-8746-43b1-83cf-733293873d04} TCPHandler: Code: 43. DB::Exception: First argument of function wordShingleMinHash must be String, got Array(Nothing): In scope SELECT wordShingleMinHash([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:26.094495 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:26.142356 [ 2707 ] {9ae0a416-1d25-43c1-881b-935eef1ad57c} executeQuery: Code: 43. DB::Exception: First argument of function wordShingleMinHashArg must be String, got Array(Nothing): In scope SELECT wordShingleMinHashArg([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT wordShingleMinHashArg([], [], []);) 2025.09.25 21:12:26.148648 [ 2707 ] {9ae0a416-1d25-43c1-881b-935eef1ad57c} TCPHandler: Code: 43. DB::Exception: First argument of function wordShingleMinHashArg must be String, got Array(Nothing): In scope SELECT wordShingleMinHashArg([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:26.199581 [ 2707 ] {c73e9a51-8c4e-4cd3-a5ff-9a005f69fe96} executeQuery: Code: 43. DB::Exception: First argument of function wordShingleMinHashArgCaseInsensitive must be String, got Array(Nothing): In scope SELECT wordShingleMinHashArgCaseInsensitive([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT wordShingleMinHashArgCaseInsensitive([], [], []);) 2025.09.25 21:12:26.209351 [ 2707 ] {c73e9a51-8c4e-4cd3-a5ff-9a005f69fe96} TCPHandler: Code: 43. DB::Exception: First argument of function wordShingleMinHashArgCaseInsensitive must be String, got Array(Nothing): In scope SELECT wordShingleMinHashArgCaseInsensitive([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:26.285805 [ 2707 ] {7ba6e043-4908-4348-a9da-11d98bbf2198} executeQuery: Code: 43. DB::Exception: First argument of function wordShingleMinHashArgCaseInsensitiveUTF8 must be String, got Array(Nothing): In scope SELECT wordShingleMinHashArgCaseInsensitiveUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT wordShingleMinHashArgCaseInsensitiveUTF8([], [], []);) 2025.09.25 21:12:26.289394 [ 2707 ] {7ba6e043-4908-4348-a9da-11d98bbf2198} TCPHandler: Code: 43. DB::Exception: First argument of function wordShingleMinHashArgCaseInsensitiveUTF8 must be String, got Array(Nothing): In scope SELECT wordShingleMinHashArgCaseInsensitiveUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:26.344709 [ 2707 ] {58f7bd7b-5606-40e9-b773-a7314e363826} executeQuery: Code: 43. DB::Exception: First argument of function wordShingleMinHashArgUTF8 must be String, got Array(Nothing): In scope SELECT wordShingleMinHashArgUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT wordShingleMinHashArgUTF8([], [], []);) 2025.09.25 21:12:26.350491 [ 2707 ] {58f7bd7b-5606-40e9-b773-a7314e363826} TCPHandler: Code: 43. DB::Exception: First argument of function wordShingleMinHashArgUTF8 must be String, got Array(Nothing): In scope SELECT wordShingleMinHashArgUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:26.420593 [ 2707 ] {5d2670ff-fa55-4532-8f28-57cbdc40be8f} executeQuery: Code: 43. DB::Exception: First argument of function wordShingleMinHashCaseInsensitive must be String, got Array(Nothing): In scope SELECT wordShingleMinHashCaseInsensitive([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT wordShingleMinHashCaseInsensitive([], [], []);) 2025.09.25 21:12:26.430766 [ 2707 ] {5d2670ff-fa55-4532-8f28-57cbdc40be8f} TCPHandler: Code: 43. DB::Exception: First argument of function wordShingleMinHashCaseInsensitive must be String, got Array(Nothing): In scope SELECT wordShingleMinHashCaseInsensitive([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:26.507963 [ 2707 ] {41277002-7b6e-4565-946b-3220aa4bfaaf} executeQuery: Code: 43. DB::Exception: First argument of function wordShingleMinHashCaseInsensitiveUTF8 must be String, got Array(Nothing): In scope SELECT wordShingleMinHashCaseInsensitiveUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT wordShingleMinHashCaseInsensitiveUTF8([], [], []);) 2025.09.25 21:12:26.511566 [ 2707 ] {41277002-7b6e-4565-946b-3220aa4bfaaf} TCPHandler: Code: 43. DB::Exception: First argument of function wordShingleMinHashCaseInsensitiveUTF8 must be String, got Array(Nothing): In scope SELECT wordShingleMinHashCaseInsensitiveUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:26.677490 [ 2707 ] {8efbfa6e-9e39-4fb8-955e-4e607c46482c} executeQuery: Code: 43. DB::Exception: First argument of function wordShingleMinHashUTF8 must be String, got Array(Nothing): In scope SELECT wordShingleMinHashUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT wordShingleMinHashUTF8([], [], []);) 2025.09.25 21:12:26.681466 [ 2707 ] {8efbfa6e-9e39-4fb8-955e-4e607c46482c} TCPHandler: Code: 43. DB::Exception: First argument of function wordShingleMinHashUTF8 must be String, got Array(Nothing): In scope SELECT wordShingleMinHashUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:26.751930 [ 2707 ] {ee95beec-6a69-4d64-9f2d-677ad42d6cd5} executeQuery: Code: 43. DB::Exception: First argument of function wordShingleSimHash must be String, got Array(Nothing): In scope SELECT wordShingleSimHash([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT wordShingleSimHash([], [], []);) 2025.09.25 21:12:26.759818 [ 3360 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {02260661-95d0-4eca-a73d-2543d99ce5b9::202509_113_118_1}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:26.760325 [ 2707 ] {ee95beec-6a69-4d64-9f2d-677ad42d6cd5} TCPHandler: Code: 43. DB::Exception: First argument of function wordShingleSimHash must be String, got Array(Nothing): In scope SELECT wordShingleSimHash([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:26.789090 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:26.803040 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:26.808478 [ 2707 ] {63bf960d-3972-45cc-b376-03b98b5e44d3} executeQuery: Code: 43. DB::Exception: First argument of function wordShingleSimHashCaseInsensitive must be String, got Array(Nothing): In scope SELECT wordShingleSimHashCaseInsensitive([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT wordShingleSimHashCaseInsensitive([], [], []);) 2025.09.25 21:12:26.815221 [ 2707 ] {63bf960d-3972-45cc-b376-03b98b5e44d3} TCPHandler: Code: 43. DB::Exception: First argument of function wordShingleSimHashCaseInsensitive must be String, got Array(Nothing): In scope SELECT wordShingleSimHashCaseInsensitive([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:26.881847 [ 2707 ] {4f155cf6-8d87-4e4f-bd33-0bdff09e7653} executeQuery: Code: 43. DB::Exception: First argument of function wordShingleSimHashCaseInsensitiveUTF8 must be String, got Array(Nothing): In scope SELECT wordShingleSimHashCaseInsensitiveUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT wordShingleSimHashCaseInsensitiveUTF8([], [], []);) 2025.09.25 21:12:26.891197 [ 2707 ] {4f155cf6-8d87-4e4f-bd33-0bdff09e7653} TCPHandler: Code: 43. DB::Exception: First argument of function wordShingleSimHashCaseInsensitiveUTF8 must be String, got Array(Nothing): In scope SELECT wordShingleSimHashCaseInsensitiveUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:26.968865 [ 2707 ] {f126fbb3-525a-4a4c-8603-399277412a6c} executeQuery: Code: 43. DB::Exception: First argument of function wordShingleSimHashUTF8 must be String, got Array(Nothing): In scope SELECT wordShingleSimHashUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT wordShingleSimHashUTF8([], [], []);) 2025.09.25 21:12:26.982767 [ 2707 ] {f126fbb3-525a-4a4c-8603-399277412a6c} TCPHandler: Code: 43. DB::Exception: First argument of function wordShingleSimHashUTF8 must be String, got Array(Nothing): In scope SELECT wordShingleSimHashUTF8([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:27.139771 [ 2707 ] {02e5a769-ec3e-4b95-ae8a-2c19032292d3} executeQuery: Code: 43. DB::Exception: Illegal type (Array(Nothing)) of 1 argument of function xor: In scope SELECT xor([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT xor([], [], []);) 2025.09.25 21:12:27.145447 [ 2707 ] {02e5a769-ec3e-4b95-ae8a-2c19032292d3} TCPHandler: Code: 43. DB::Exception: Illegal type (Array(Nothing)) of 1 argument of function xor: In scope SELECT xor([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:27.555115 [ 2707 ] {355210ed-27a6-4529-bb64-c116e5c84ae7} executeQuery: Code: 42. DB::Exception: Number of arguments for function kostikConsistentHash doesn't match: passed 3, should be 2: In scope SELECT yandexConsistentHash([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT yandexConsistentHash([], [], []);) 2025.09.25 21:12:27.565640 [ 2707 ] {355210ed-27a6-4529-bb64-c116e5c84ae7} TCPHandler: Code: 42. DB::Exception: Number of arguments for function kostikConsistentHash doesn't match: passed 3, should be 2: In scope SELECT yandexConsistentHash([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:27.687600 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:27.736795 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:27.742266 [ 2707 ] {d25776fa-f320-49e1-83f7-8f5fb5996088} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function toYearWeek. Must be Date, Date32, DateTime or DateTime64: In scope SELECT yearweek([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT yearweek([], [], []);) 2025.09.25 21:12:27.751555 [ 2707 ] {d25776fa-f320-49e1-83f7-8f5fb5996088} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function toYearWeek. Must be Date, Date32, DateTime or DateTime64: In scope SELECT yearweek([], [], []). (ILLEGAL_TYPE_OF_ARGUMENT) 2025.09.25 21:12:27.817460 [ 2707 ] {1e434db7-63e0-44c5-b16f-362e5fbe0d37} executeQuery: Code: 42. DB::Exception: Number of arguments for function yesterday doesn't match: passed 3, should be 0: In scope SELECT yesterday([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT yesterday([], [], []);) 2025.09.25 21:12:27.823866 [ 2707 ] {1e434db7-63e0-44c5-b16f-362e5fbe0d37} TCPHandler: Code: 42. DB::Exception: Number of arguments for function yesterday doesn't match: passed 3, should be 0: In scope SELECT yesterday([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:27.901290 [ 2707 ] {e2220596-c857-404f-8630-fa7557fd922b} executeQuery: Code: 42. DB::Exception: Number of arguments for function zookeeperSessionUptime doesn't match: passed 3, should be 0: In scope SELECT zookeeperSessionUptime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34710) (comment: 00534_functions_bad_arguments12.sh) (in query: SELECT zookeeperSessionUptime([], [], []);) 2025.09.25 21:12:27.905426 [ 2707 ] {e2220596-c857-404f-8630-fa7557fd922b} TCPHandler: Code: 42. DB::Exception: Number of arguments for function zookeeperSessionUptime doesn't match: passed 3, should be 0: In scope SELECT zookeeperSessionUptime([], [], []). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) 2025.09.25 21:12:28.835094 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:28.849616 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:29.865428 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:29.932778 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:30.202223 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:30.215730 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:30.987945 [ 3682 ] {9129a5e4-0b8c-4e75-baf0-d9b431a02c34} executeQuery: Code: 43. DB::Exception: Illegal type Array(UInt8) of argument of function toUInt64: In scope SELECT dictGet('02176_test_simple_key_dictionary', 'value', [0]). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53308) (comment: 02176_dict_get_has_implicit_key_cast.sql) (in query: SELECT dictGet('02176_test_simple_key_dictionary', 'value', [0]);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionConvert, DB::(anonymous namespace)::NameToUInt64, DB::(anonymous namespace)::ToNumberMonotonicity>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000ade86f7 5. DB::FunctionToExecutableFunctionAdaptor::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac9135 6. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccce5b 7. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 8. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 9. DB::IFunctionBase::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000aac7d8f 10. COW::immutable_ptr std::__function::__policy_invoker::immutable_ptr (std::vector>&, std::shared_ptr const&, DB::ColumnNullable const*, unsigned long)>::__call_impl, std::shared_ptr const&)::'lambda'(std::vector>&, std::shared_ptr const&, DB::ColumnNullable const*, unsigned long), COW::immutable_ptr (std::vector>&, std::shared_ptr const&, DB::ColumnNullable const*, unsigned long)>>(std::__function::__policy_storage const*, std::vector>&, std::shared_ptr const&, DB::ColumnNullable const*, unsigned long) @ 0x000000000ad4b0e1 11. DB::(anonymous namespace)::ExecutableFunctionCast::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000acf2e8b 12. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccce5b 13. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce7e4 14. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 15. DB::IFunctionBase::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000aac7d8f 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::castColumn(DB::CastType, DB::ColumnWithTypeAndName const&, std::shared_ptr const&, DB::InternalCastFunctionCache*) @ 0x0000000029a9f39d 17. ./build_docker/./src/Interpreters/castColumn.cpp:49: DB::castColumnAccurate(DB::ColumnWithTypeAndName const&, std::shared_ptr const&, DB::InternalCastFunctionCache*) @ 0x0000000029aa01dc 18. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::IDictionary::convertKeyColumns(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&, std::vector, std::allocator>>&) const @ 0x000000001b64f8ed 19. DB::FunctionDictGetNoType<(DB::DictionaryGetFunctionType)0>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000b5d1275 20. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 21. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 22. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 23. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 24. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 25. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 26. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 27. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 28. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 29. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 30. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 31. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 2025.09.25 21:12:30.991136 [ 3682 ] {9129a5e4-0b8c-4e75-baf0-d9b431a02c34} TCPHandler: Code: 43. DB::Exception: Illegal type Array(UInt8) of argument of function toUInt64: In scope SELECT dictGet('02176_test_simple_key_dictionary', 'value', [0]). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionConvert, DB::(anonymous namespace)::NameToUInt64, DB::(anonymous namespace)::ToNumberMonotonicity>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000ade86f7 5. DB::FunctionToExecutableFunctionAdaptor::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac9135 6. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccce5b 7. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 8. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 9. DB::IFunctionBase::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000aac7d8f 10. COW::immutable_ptr std::__function::__policy_invoker::immutable_ptr (std::vector>&, std::shared_ptr const&, DB::ColumnNullable const*, unsigned long)>::__call_impl, std::shared_ptr const&)::'lambda'(std::vector>&, std::shared_ptr const&, DB::ColumnNullable const*, unsigned long), COW::immutable_ptr (std::vector>&, std::shared_ptr const&, DB::ColumnNullable const*, unsigned long)>>(std::__function::__policy_storage const*, std::vector>&, std::shared_ptr const&, DB::ColumnNullable const*, unsigned long) @ 0x000000000ad4b0e1 11. DB::(anonymous namespace)::ExecutableFunctionCast::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000acf2e8b 12. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccce5b 13. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce7e4 14. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 15. DB::IFunctionBase::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000aac7d8f 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::castColumn(DB::CastType, DB::ColumnWithTypeAndName const&, std::shared_ptr const&, DB::InternalCastFunctionCache*) @ 0x0000000029a9f39d 17. ./build_docker/./src/Interpreters/castColumn.cpp:49: DB::castColumnAccurate(DB::ColumnWithTypeAndName const&, std::shared_ptr const&, DB::InternalCastFunctionCache*) @ 0x0000000029aa01dc 18. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::IDictionary::convertKeyColumns(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&, std::vector, std::allocator>>&) const @ 0x000000001b64f8ed 19. DB::FunctionDictGetNoType<(DB::DictionaryGetFunctionType)0>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000b5d1275 20. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 21. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 22. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 23. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 24. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 25. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 26. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 27. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 28. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 29. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 30. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 31. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 2025.09.25 21:12:31.185611 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:31.199697 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:32.047523 [ 3682 ] {118d0e2d-07e1-44c8-81fd-3e33f91f64c4} executeQuery: Code: 43. DB::Exception: Illegal type Array(UInt8) of argument of function toUInt64: In scope SELECT dictHas('02176_test_simple_key_dictionary', [0]). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53308) (comment: 02176_dict_get_has_implicit_key_cast.sql) (in query: SELECT dictHas('02176_test_simple_key_dictionary', [0]);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionConvert, DB::(anonymous namespace)::NameToUInt64, DB::(anonymous namespace)::ToNumberMonotonicity>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000ade86f7 5. DB::FunctionToExecutableFunctionAdaptor::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac9135 6. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccce5b 7. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 8. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 9. DB::IFunctionBase::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000aac7d8f 10. COW::immutable_ptr std::__function::__policy_invoker::immutable_ptr (std::vector>&, std::shared_ptr const&, DB::ColumnNullable const*, unsigned long)>::__call_impl, std::shared_ptr const&)::'lambda'(std::vector>&, std::shared_ptr const&, DB::ColumnNullable const*, unsigned long), COW::immutable_ptr (std::vector>&, std::shared_ptr const&, DB::ColumnNullable const*, unsigned long)>>(std::__function::__policy_storage const*, std::vector>&, std::shared_ptr const&, DB::ColumnNullable const*, unsigned long) @ 0x000000000ad4b0e1 11. DB::(anonymous namespace)::ExecutableFunctionCast::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000acf2e8b 12. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccce5b 13. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce7e4 14. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 15. DB::IFunctionBase::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000aac7d8f 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::castColumn(DB::CastType, DB::ColumnWithTypeAndName const&, std::shared_ptr const&, DB::InternalCastFunctionCache*) @ 0x0000000029a9f39d 17. ./build_docker/./src/Interpreters/castColumn.cpp:49: DB::castColumnAccurate(DB::ColumnWithTypeAndName const&, std::shared_ptr const&, DB::InternalCastFunctionCache*) @ 0x0000000029aa01dc 18. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::IDictionary::convertKeyColumns(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&, std::vector, std::allocator>>&) const @ 0x000000001b64f8ed 19. DB::FunctionDictHas::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000b5ee4a9 20. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 21. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 22. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 23. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 24. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 25. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 26. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 27. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 28. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 29. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 30. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 31. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 2025.09.25 21:12:32.050503 [ 3682 ] {118d0e2d-07e1-44c8-81fd-3e33f91f64c4} TCPHandler: Code: 43. DB::Exception: Illegal type Array(UInt8) of argument of function toUInt64: In scope SELECT dictHas('02176_test_simple_key_dictionary', [0]). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::(anonymous namespace)::FunctionConvert, DB::(anonymous namespace)::NameToUInt64, DB::(anonymous namespace)::ToNumberMonotonicity>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000ade86f7 5. DB::FunctionToExecutableFunctionAdaptor::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac9135 6. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccce5b 7. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 8. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 9. DB::IFunctionBase::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000aac7d8f 10. COW::immutable_ptr std::__function::__policy_invoker::immutable_ptr (std::vector>&, std::shared_ptr const&, DB::ColumnNullable const*, unsigned long)>::__call_impl, std::shared_ptr const&)::'lambda'(std::vector>&, std::shared_ptr const&, DB::ColumnNullable const*, unsigned long), COW::immutable_ptr (std::vector>&, std::shared_ptr const&, DB::ColumnNullable const*, unsigned long)>>(std::__function::__policy_storage const*, std::vector>&, std::shared_ptr const&, DB::ColumnNullable const*, unsigned long) @ 0x000000000ad4b0e1 11. DB::(anonymous namespace)::ExecutableFunctionCast::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000acf2e8b 12. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccce5b 13. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce7e4 14. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 15. DB::IFunctionBase::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000aac7d8f 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::castColumn(DB::CastType, DB::ColumnWithTypeAndName const&, std::shared_ptr const&, DB::InternalCastFunctionCache*) @ 0x0000000029a9f39d 17. ./build_docker/./src/Interpreters/castColumn.cpp:49: DB::castColumnAccurate(DB::ColumnWithTypeAndName const&, std::shared_ptr const&, DB::InternalCastFunctionCache*) @ 0x0000000029aa01dc 18. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::IDictionary::convertKeyColumns(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&, std::vector, std::allocator>>&) const @ 0x000000001b64f8ed 19. DB::FunctionDictHas::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000b5ee4a9 20. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 21. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 22. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 23. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 24. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 25. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 26. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 27. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 28. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 29. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 30. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 31. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 2025.09.25 21:12:32.116952 [ 3682 ] {6c02a545-437d-4b24-91fd-b84c785e720b} executeQuery: Code: 62. DB::Exception: Cannot TRUNCATE dictionary. (SYNTAX_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53308) (comment: 02176_dict_get_has_implicit_key_cast.sql) (in query: DROP DICTIONARY 02176_test_simple_key_dictionary;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:262: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f1adc 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:32.119337 [ 3682 ] {6c02a545-437d-4b24-91fd-b84c785e720b} TCPHandler: Code: 62. DB::Exception: Cannot TRUNCATE dictionary. (SYNTAX_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:262: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f1adc 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:32.258201 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:32.276875 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:33.109074 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:33.126944 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:33.781581 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:33.981043 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:34.733110 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:34.760605 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:35.142849 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:35.161480 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:35.421154 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:35.514820 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:35.744462 [ 7168 ] {a5269dfb-3cb0-480f-91c4-a2c935ee5204} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=729). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53352) (comment: 01904_dictionary_default_nullable_type.sql) (in query: SELECT dictGet('flat_dictionary', 'value', toUInt64(1));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:35.750653 [ 7168 ] {a5269dfb-3cb0-480f-91c4-a2c935ee5204} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=729). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:36.285268 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:36.299005 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:37.072256 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:37.110853 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:37.140364 [ 7168 ] {850c6820-63c8-4c1d-981a-58e7dac17a17} executeQuery: Code: 452. DB::Exception: Setting allow_experimental_analyzer should not be changed. (SETTING_CONSTRAINT_VIOLATION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46406) (comment: 00757_enum_defaults_const.sql) (in query: SET enable_analyzer=0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges const&, DB::SettingSource) const @ 0x0000000026f807e3 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraints(DB::SettingsChanges const&, DB::SettingSource) @ 0x000000002853b6ad 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::execute() @ 0x00000000294f603b 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:37.158926 [ 7168 ] {850c6820-63c8-4c1d-981a-58e7dac17a17} TCPHandler: Code: 452. DB::Exception: Setting allow_experimental_analyzer should not be changed. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges const&, DB::SettingSource) const @ 0x0000000026f807e3 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraints(DB::SettingsChanges const&, DB::SettingSource) @ 0x000000002853b6ad 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::execute() @ 0x00000000294f603b 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:38.015692 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:38.045005 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:38.624945 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:38.669598 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:39.577372 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:39.611606 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:40.212685 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:40.299846 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:40.719986 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:40.780761 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:41.297015 [ 8396 ] {9248e8c6-2fe7-48c4-94cc-b2f78429232a} executeQuery: Code: 57. DB::Exception: Table test_3.test already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46432) (comment: 02903_client_insert_in_background.sh) (in query: create table test (x UInt64) engine=Memory), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:41.303439 [ 8396 ] {9248e8c6-2fe7-48c4-94cc-b2f78429232a} TCPHandler: Code: 57. DB::Exception: Table test_3.test already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:41.514182 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:41.592691 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:41.917571 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:41.980800 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:43.044892 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:43.080153 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:44.089583 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:44.110954 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:44.447267 [ 8396 ] {39bd5495-0fac-4a26-9c19-fc182261787b} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=729). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46446) (comment: 00181_aggregate_functions_statistics_stable.sql) (in query: /* varSampStable */ SELECT varSampStable(x_value) FROM (SELECT x_value FROM series LIMIT 0);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:44.453518 [ 8396 ] {39bd5495-0fac-4a26-9c19-fc182261787b} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=729). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:44.577463 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:44.613370 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:45.014747 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:45.035725 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:45.435974 [ 7168 ] {68d62433-6beb-488e-bd05-63a88ab5e37d} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=730). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46426) (comment: 00441_nulls_in.sql) (in query: SELECT (number, -nullIf(number, 3)) IN (SELECT nullIf(number, 2), -number FROM system.numbers LIMIT 1, 3) AS res FROM system.numbers LIMIT 5;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:45.442713 [ 7168 ] {68d62433-6beb-488e-bd05-63a88ab5e37d} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=730). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:46.095209 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:46.111447 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:46.581335 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:46.612152 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:47.241964 [ 8396 ] {75e7eaf3-bc6e-4347-87c4-7346370f095b} executeQuery: Code: 440. DB::Exception: LIMIT expression must be constant with numeric type. Actual arrayJoin([1, 2]). In scope SELECT * FROM system.numbers LIMIT arrayJoin([1, 2]). (INVALID_LIMIT_EXPRESSION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46454) (comment: 01165_lost_part_empty_partition.sql) (in query: insert into rmt1(n) select * from system.numbers limit arrayJoin([1, 2]);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String const&, String&&, String&&) @ 0x000000000b578ce0 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:792: DB::QueryAnalyzer::convertLimitOffsetExpression(std::shared_ptr&, String const&, DB::IdentifierResolveScope&) @ 0x000000002837e4b4 5. ./contrib/llvm-project/libcxx/include/string:1499: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835bba0 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: DB::InterpreterInsertQuery::buildInsertSelectPipeline(DB::ASTInsertQuery&, std::shared_ptr) @ 0x00000000293547ba 12. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:789: DB::InterpreterInsertQuery::execute() @ 0x000000002935b5a0 13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:47.245271 [ 8396 ] {75e7eaf3-bc6e-4347-87c4-7346370f095b} TCPHandler: Code: 440. DB::Exception: LIMIT expression must be constant with numeric type. Actual arrayJoin([1, 2]). In scope SELECT * FROM system.numbers LIMIT arrayJoin([1, 2]). (INVALID_LIMIT_EXPRESSION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String const&, String&&, String&&) @ 0x000000000b578ce0 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:792: DB::QueryAnalyzer::convertLimitOffsetExpression(std::shared_ptr&, String const&, DB::IdentifierResolveScope&) @ 0x000000002837e4b4 5. ./contrib/llvm-project/libcxx/include/string:1499: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835bba0 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: DB::InterpreterInsertQuery::buildInsertSelectPipeline(DB::ASTInsertQuery&, std::shared_ptr) @ 0x00000000293547ba 12. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:789: DB::InterpreterInsertQuery::execute() @ 0x000000002935b5a0 13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:47.279296 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:47.370910 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:47.463676 [ 3682 ] {e2b48657-b574-46dd-909f-fc3291acf61d} executeQuery: Code: 48. DB::Exception: Table engine Join supports only DELETE mutations. (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46466) (comment: 01821_join_table_mutation.sql) (in query: ALTER TABLE join_table_mutation UPDATE name = 'some' WHERE 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Storages/StorageJoin.cpp:165: DB::StorageJoin::checkMutationIsPossible(DB::MutationCommands const&, DB::Settings const&) const @ 0x000000002b82b3e5 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000029174782 6. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:47.467821 [ 3682 ] {e2b48657-b574-46dd-909f-fc3291acf61d} TCPHandler: Code: 48. DB::Exception: Table engine Join supports only DELETE mutations. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Storages/StorageJoin.cpp:165: DB::StorageJoin::checkMutationIsPossible(DB::MutationCommands const&, DB::Settings const&) const @ 0x000000002b82b3e5 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000029174782 6. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:47.855866 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:47.988219 [ 7168 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:48.267073 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:48.283909 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:48.577806 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:49.322384 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:49.337546 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:49.596566 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:50.169208 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:50.198374 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:50.654631 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:50.752508 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:50.845715 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:51.499962 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:51.807634 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:51.839723 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:52.319079 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:52.541063 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:52.562812 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:52.753203 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:52.863259 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:52.877306 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:53.539139 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:53.617977 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:53.738671 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:54.401462 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:54.791498 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:54.827376 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:55.307743 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:55.741370 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:55.756641 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:56.080090 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:56.207039 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:56.943634 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:56.959218 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:57.030522 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:57.307267 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:57.359527 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:57.370322 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:57.675864 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:57.689785 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:57.885478 [ 3682 ] {6a64179f-6feb-49ec-83b6-3f34c27e1d8c} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:39372) (comment: 01509_check_many_parallel_quorum_inserts_long.sh) (in query: CREATE TABLE r5 (`x` UInt64) ENGINE = ReplicatedMergeTree('/clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many', 'r5') ORDER BY x SETTINGS index_granularity = 21481, min_bytes_for_wide_part = 0, ratio_of_defaults_for_sparse_serialization = 0., replace_long_file_name_to_hash = false, max_file_name_length = 128, min_bytes_for_full_part_storage = 0, compact_parts_max_bytes_to_buffer = 461584303, compact_parts_max_granules_to_buffer = 191, compact_parts_merge_max_bytes_to_prefetch_part = 31082813, merge_max_block_size = 4640, old_parts_lifetime = 164., prefer_fetch_merged_part_size_threshold = 10737418240, vertical_merge_algorithm_min_rows_to_activate = 1000000, vertical_merge_algorithm_min_columns_to_activate = 100, min_merge_bytes_to_use_direct_io = 10737418240, index_granularity_bytes = 22918056, concurrent_part_removal_threshold = 100, allow_vertical_merges_from_compact_to_wide_parts = false, cache_populated_by_fetch = true, marks_compress_block_size = 28470, primary_key_compress_block_size = 11571), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c45d4f6 8. ./contrib/llvm-project/libcxx/include/optional:260: DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, DB::LoadingStrictnessLevel, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>, DB::RenamingRestrictions, bool) @ 0x000000002ba16434 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1460: std::shared_ptr std::allocate_shared[abi:v15007], String&, String&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>, DB::RenamingRestrictions&, bool&, void>(std::allocator const&, String&, String&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>&&, DB::RenamingRestrictions&, bool&) @ 0x000000002cbab63b 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::create(DB::StorageFactory::Arguments const&) @ 0x000000002cba1fff 11. ./build_docker/./src/Storages/StorageFactory.cpp:225: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x000000002b77bf53 12. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1659: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291b9254 13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:57.903572 [ 3682 ] {6a64179f-6feb-49ec-83b6-3f34c27e1d8c} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c45d4f6 8. ./contrib/llvm-project/libcxx/include/optional:260: DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, DB::LoadingStrictnessLevel, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>, DB::RenamingRestrictions, bool) @ 0x000000002ba16434 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1460: std::shared_ptr std::allocate_shared[abi:v15007], String&, String&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>, DB::RenamingRestrictions&, bool&, void>(std::allocator const&, String&, String&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>&&, DB::RenamingRestrictions&, bool&) @ 0x000000002cbab63b 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::create(DB::StorageFactory::Arguments const&) @ 0x000000002cba1fff 11. ./build_docker/./src/Storages/StorageFactory.cpp:225: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x000000002b77bf53 12. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1659: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291b9254 13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:12:58.215317 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:58.229426 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:58.283036 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:58.619827 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:58.695365 [ 2966 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:58.752155 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:58.769325 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:59.173162 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:59.572457 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:59.589142 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:59.699402 [ 3000 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:59.734805 [ 2998 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:12:59.741376 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:00.043529 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:00.632240 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:00.654224 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:00.702945 [ 3024 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:00.738654 [ 3026 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:01.118935 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:01.131849 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:01.143966 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:01.285808 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:01.706980 [ 3052 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:01.751172 [ 3056 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:02.228032 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:02.253132 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:02.366372 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:02.709240 [ 3077 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:02.753985 [ 3079 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:02.760439 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:03.254986 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:03.267864 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:03.698763 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:03.711305 [ 3103 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:03.756300 [ 3102 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:04.325381 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:04.459949 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:04.475628 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:04.714511 [ 3125 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:04.758316 [ 3137 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:05.216429 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:05.242001 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:05.257777 [ 3352 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {fffcce7c-f4e8-42ed-b1c9-22aa3ca82e93::202509_1_68_28}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001adfc17a 4. ./build_docker/./src/Common/MemoryTracker.cpp:187: MemoryTracker::injectFault() const @ 0x000000001adf8d1a 5. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x000000002cc30a65 6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f47b 7. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:05.267744 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:05.495453 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:05.507913 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:05.718888 [ 3163 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:05.761691 [ 3170 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:05.783987 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:05.795755 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:05.968184 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:06.201153 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:06.413690 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:06.722085 [ 3210 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:06.764730 [ 3217 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:06.764876 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:06.884725 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:07.062693 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:07.768382 [ 3233 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:07.790787 [ 3230 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:07.973312 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:08.152546 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:08.187213 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:08.772388 [ 2746 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:08.795146 [ 2747 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:09.007502 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:09.045265 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:09.053215 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:09.250316 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:09.779822 [ 2775 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:09.799041 [ 2776 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:09.871052 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:09.903623 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:09.923560 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:10.638648 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:10.754130 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:10.771985 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:10.782827 [ 2797 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:10.802871 [ 2798 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:10.884145 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:11.788144 [ 2828 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:11.806539 [ 2807 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:11.869962 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:12.060367 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:12.154206 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:12.793192 [ 2853 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:12.810038 [ 2850 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:12.984683 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:12.986724 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:13.021215 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:13.437319 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:13.438025 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:13.460840 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:13.796345 [ 2881 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:13.813467 [ 2882 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:14.000595 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:14.190928 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:14.237686 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:14.798873 [ 2905 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:14.863592 [ 2908 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:14.960710 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:15.081093 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:15.090201 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:15.258680 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:15.442243 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:15.815901 [ 2924 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:15.866809 [ 2934 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:16.038259 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:16.214684 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:16.285139 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:16.293030 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:16.403110 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:16.819428 [ 2961 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:16.866872 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:16.870232 [ 2965 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:16.892923 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:17.314175 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:17.512863 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:17.526149 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:17.553223 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:17.822806 [ 2992 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:17.873519 [ 2991 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:18.394544 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:18.744630 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:18.770948 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:18.827375 [ 3004 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:18.856992 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:18.876832 [ 3011 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:19.638797 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:19.830801 [ 3033 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:19.880146 [ 3032 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:19.960412 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:19.990015 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:20.137237 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:20.286090 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:20.323459 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:20.833306 [ 3054 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:20.882608 [ 3053 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:21.082392 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:21.218332 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:21.248631 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:21.544789 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:21.800321 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:21.823160 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:21.836751 [ 3088 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:21.885759 [ 3092 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:22.045510 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:22.294821 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:22.321167 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:22.694244 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:22.710281 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:22.714215 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:22.839940 [ 3117 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:22.889371 [ 3119 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:23.148453 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:23.609444 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:23.634293 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:23.833109 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:23.843380 [ 3139 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:23.899939 [ 3140 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:23.970303 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:23.987583 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:24.592353 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:24.624171 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:24.823146 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:24.846795 [ 3159 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:24.903782 [ 3170 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:25.400280 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:25.416929 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:25.640394 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:25.674459 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:25.723367 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:25.858117 [ 3200 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:25.906357 [ 3193 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:25.931439 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:25.946533 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:26.416367 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:26.426264 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:26.825331 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:26.860028 [ 3227 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:26.908401 [ 3233 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:27.184739 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:27.200179 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:27.653484 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:27.844102 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:27.856557 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:27.861772 [ 3254 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:27.910359 [ 3260 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:28.285230 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:28.513630 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:28.522865 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:28.865509 [ 2772 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:28.917295 [ 2779 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:28.982715 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:29.309655 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:29.344228 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:29.867513 [ 2777 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:29.919375 [ 2750 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:30.025605 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:30.031266 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:30.074130 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:30.238340 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:30.484970 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:30.497669 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:30.792640 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:30.800518 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:30.869703 [ 2809 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:30.921498 [ 2814 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:30.966758 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:31.670940 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:31.871834 [ 2832 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:31.923510 [ 2836 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:31.983674 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:31.988968 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:31.998942 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:32.825905 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:32.873773 [ 2862 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:32.925397 [ 2858 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:33.022425 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:33.032069 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:33.256953 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:33.268321 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:33.589416 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:33.616960 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:33.835925 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:33.864791 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:33.876998 [ 2884 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:33.928599 [ 2886 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:33.958775 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:34.367347 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:34.395045 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:34.852156 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:34.867176 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:34.880116 [ 2908 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:34.932672 [ 2916 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:35.036910 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:35.293711 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:35.883353 [ 2931 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:35.936632 [ 2935 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:36.028694 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:36.051515 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:36.249173 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:36.612500 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:36.629372 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:36.808575 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:36.896229 [ 2963 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:36.983196 [ 2962 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:37.407645 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:37.890494 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:37.899830 [ 2986 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:37.916544 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:37.986640 [ 2989 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:38.172947 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:38.645439 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:38.829515 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:38.856433 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:38.903435 [ 3006 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:38.990226 [ 3007 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:39.076886 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:39.123407 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:39.160813 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:39.474548 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:39.908373 [ 3028 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:39.993618 [ 3029 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:40.118178 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:40.170334 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:40.214361 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:40.515695 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:40.706927 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:40.911698 [ 3052 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:40.997053 [ 3055 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:41.057220 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:41.072590 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:41.540299 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:41.691845 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:41.913963 [ 3086 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:41.951345 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:41.953206 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:41.963522 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:41.999284 [ 3085 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:42.307689 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:42.449871 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:42.620866 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:42.630695 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:42.916524 [ 3112 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:42.932259 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:43.005676 [ 3115 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:43.617874 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:43.635056 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:43.679011 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:43.919681 [ 3134 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:44.008833 [ 3137 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:44.649200 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:44.714998 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:44.729868 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:44.923172 [ 3142 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:45.016423 [ 3158 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:45.337733 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:45.545915 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:45.596725 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:45.602180 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:45.773920 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:45.907082 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:45.926459 [ 3183 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:46.019623 [ 3188 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:46.720434 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:46.748974 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:46.760932 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:46.929741 [ 3210 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:47.022928 [ 3215 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:47.123559 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:47.141415 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:47.313815 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:47.635246 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:47.933491 [ 3237 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:48.029451 [ 3238 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:48.031706 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:48.076630 [ 3682 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:48.097230 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:48.458212 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:48.938206 [ 3256 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:49.032691 [ 3254 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:49.125039 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:49.275578 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:49.301115 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:49.679824 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:49.940803 [ 2769 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:50.035518 [ 2779 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:50.128591 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:50.363387 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:50.375429 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:50.511760 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:50.694741 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:50.725418 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:50.944467 [ 2787 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:51.037850 [ 2800 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:51.068957 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:51.719187 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:51.742441 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:51.852524 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:51.947974 [ 2807 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:52.041144 [ 2825 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:52.661220 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:52.808502 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:52.863096 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:52.872114 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:52.951682 [ 2849 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:53.044529 [ 2852 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:53.453046 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:53.487671 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:53.955399 [ 2872 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:53.958273 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:54.101184 [ 2876 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:54.619053 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:54.619541 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:54.673458 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:54.709931 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:54.958906 [ 2897 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:55.104560 [ 2902 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:55.254761 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:55.271764 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:55.637842 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:55.845562 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:55.961705 [ 2918 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:56.158654 [ 2919 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:56.307699 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:56.333339 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:56.647101 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:56.712361 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:56.979535 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:57.059583 [ 2948 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:57.162364 [ 2947 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:57.374157 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:57.388215 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:58.063508 [ 2964 ] {} test_bzms6ra8.r2 (74798e48-672a-4e95-bdb1-4f473449c434): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r2). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:58.145961 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:58.164644 [ 2974 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:58.350746 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:58.456446 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:58.768713 [ 7168 ] {496598e0-1c26-4f4a-b50f-6bdfddd98a01} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34450) (comment: 01509_check_many_parallel_quorum_inserts_long.sh) (in query: DROP DATABASE IF EXISTS test_bzms6ra8 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:13:58.779076 [ 7168 ] {496598e0-1c26-4f4a-b50f-6bdfddd98a01} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:58.875743 [ 3682 ] {f14b78dc-08d0-44fa-91a8-bec84fc2c060} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55886) (comment: 01509_check_many_parallel_quorum_inserts_long.sh) (in query: DROP DATABASE IF EXISTS test_bzms6ra8 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:13:58.879507 [ 3682 ] {f14b78dc-08d0-44fa-91a8-bec84fc2c060} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:58.899707 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:58.931408 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:58.985549 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:59.167060 [ 3002 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:59.709298 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:59.956119 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:13:59.970847 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:00.168981 [ 3019 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:00.557492 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:00.580463 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:00.881696 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:01.136571 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:01.171422 [ 3045 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:01.173989 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:01.619336 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:01.634936 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:01.683339 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:02.078092 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:02.175152 [ 3070 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:02.323772 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:02.491960 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:02.552490 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:02.577796 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:02.842251 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:03.178014 [ 3092 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:03.649551 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:03.673443 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:03.737967 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:04.181772 [ 3116 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:04.516461 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:04.588500 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:04.754921 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:05.082935 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:05.250742 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:05.262439 [ 3150 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:05.272195 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:05.274445 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:05.751783 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:06.264355 [ 3161 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:06.389888 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:06.411159 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:06.758519 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:07.266576 [ 3183 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:07.580777 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:07.582218 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:07.591916 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:08.112042 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:08.219138 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:08.236230 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:08.269111 [ 3190 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:08.692270 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:09.076541 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:09.254011 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:09.276852 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:09.312685 [ 3239 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:09.656428 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:10.095194 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:10.108761 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:10.259694 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:10.315364 [ 2745 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:10.587620 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:10.648147 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:11.005792 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:11.015372 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:11.253999 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:11.317500 [ 2763 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:11.539598 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:11.553498 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:11.934520 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:12.320982 [ 2783 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:12.487147 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:12.624421 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:12.650863 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:12.886485 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:13.188221 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:13.323333 [ 2813 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:13.632520 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:13.664371 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:14.185945 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:14.205692 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:14.206589 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:14.325478 [ 2840 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:14.883114 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:15.102160 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:15.115967 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:15.328230 [ 2860 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:15.385782 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:15.593744 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:15.616233 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:15.923408 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:16.331251 [ 2884 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:16.383072 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:16.399608 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:16.790658 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:17.059646 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:17.085784 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:17.371320 [ 2909 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:17.384953 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:17.407923 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:17.464685 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:17.843168 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:18.301915 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:18.374962 [ 2944 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:18.399512 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:18.416427 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:19.059223 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:19.447675 [ 2949 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:19.602457 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:19.612595 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:20.121549 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:20.194077 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:20.220036 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:20.451383 [ 2978 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:20.802015 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:20.820705 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:21.175966 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:21.454478 [ 3005 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:21.550234 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:21.579216 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:22.014280 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:22.041121 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:22.269573 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:22.457400 [ 3033 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:22.490181 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:22.517684 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:23.104619 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:23.124791 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:23.460383 [ 3051 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:23.470406 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:24.234019 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:24.260335 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:24.326487 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:24.463582 [ 3069 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:24.634299 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:25.075276 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:25.396788 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:25.438634 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:25.466444 [ 3096 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:25.746468 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:25.896524 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:26.211125 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:26.389422 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:26.477317 [ 3119 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:26.662944 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:26.688969 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:26.927372 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:27.346598 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:27.356312 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:27.480970 [ 3154 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:27.765417 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:28.484217 [ 3172 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:28.511101 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:28.534285 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:28.772678 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:29.234550 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:29.258372 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:29.487647 [ 3195 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:29.738159 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:29.755737 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:29.850416 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:29.964561 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:30.143313 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:30.217886 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:30.491374 [ 3218 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:30.669925 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:30.982153 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:31.045849 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:31.280100 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:31.364906 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:31.495056 [ 3250 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:31.652037 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:31.676579 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:31.885515 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:32.220138 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:32.343020 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:32.374972 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:32.449030 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:32.498634 [ 2769 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:32.810177 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:33.502396 [ 2753 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:33.531713 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:33.548290 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:33.922742 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:34.121212 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:34.506147 [ 2797 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:34.607999 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:34.629979 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:34.972648 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:35.509566 [ 2819 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:35.755950 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:35.773624 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:35.968182 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:36.416037 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:36.513238 [ 2845 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:36.931140 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:36.955588 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:37.418377 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:37.477280 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:37.486637 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:37.515763 [ 2867 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:38.252817 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:38.519346 [ 2887 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:38.566240 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:38.614417 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:38.626680 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:39.414043 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:39.429654 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:39.429727 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:39.522898 [ 2906 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:40.133381 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:40.206487 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:40.222617 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:40.446080 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:40.529898 [ 2924 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:40.965803 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:40.974637 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:41.040750 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:41.204163 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:41.226997 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:41.533886 [ 2953 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:41.648007 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:41.692421 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:41.841546 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:42.360089 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:42.377159 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:42.545034 [ 2968 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:42.889554 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:43.102576 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:43.128591 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:43.547719 [ 2988 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:43.851779 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:43.876755 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:43.891237 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:44.116871 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:44.175929 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:44.550557 [ 3014 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:44.606229 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:44.624152 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:44.815618 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:45.553027 [ 3034 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:45.562791 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:45.631903 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:45.670836 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:45.948326 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:46.013150 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:46.307279 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:46.307613 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:46.331607 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:46.556033 [ 3057 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:46.768798 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:47.013119 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:47.328683 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:47.337222 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:47.558575 [ 3086 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:47.935863 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:48.352113 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:48.374892 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:48.561552 [ 3106 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:48.749205 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:48.917796 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:48.993234 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:49.048375 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:49.474403 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:49.483382 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:49.567197 [ 3127 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:49.998076 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:50.451223 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:50.473263 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:50.507806 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:50.570053 [ 3145 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:50.846522 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:50.863450 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:51.128265 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:51.331245 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:51.573798 [ 3177 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:51.948693 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:51.979627 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:52.146988 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:52.577114 [ 3169 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:52.896982 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:52.915936 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:52.952841 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:53.265552 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:53.444645 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:53.459769 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:53.580378 [ 3220 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:53.861270 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:53.915857 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:54.026069 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:54.407297 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:54.583930 [ 3241 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:54.788586 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:55.051842 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:55.256962 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:55.277056 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:55.587539 [ 2746 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:55.743482 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:55.920018 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:55.988026 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:56.344902 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:56.362754 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:56.590963 [ 2776 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:56.889023 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:57.478804 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:57.494196 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:57.593451 [ 2795 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:57.603854 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:57.921513 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:58.118481 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:58.182483 [ 3682 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:58.596716 [ 2812 ] {} test_bzms6ra8.r1 (d8283957-5cc5-4895-8ad6-4b83e57bd81c): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/01509_check_many_parallel_quorum_inserts_long_test_bzms6ra8/parallel_quorum_many/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:58.782463 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:58.816442 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:59.051422 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:59.326919 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:59.341188 [ 7168 ] {215d43f6-f033-4682-a182-cdd607925759} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55894) (comment: 01509_check_many_parallel_quorum_inserts_long.sh) (in query: DROP DATABASE IF EXISTS test_bzms6ra8 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:14:59.344426 [ 7168 ] {215d43f6-f033-4682-a182-cdd607925759} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:14:59.348689 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:00.029394 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:00.466054 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:00.515702 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:00.778558 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:01.581663 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:01.618594 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:01.908755 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:02.499853 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:02.592453 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:02.607199 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:02.607436 [ 3352 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {261a6bdf-2436-455f-b6df-dbd9b591b8be::202509_1_75_15}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:02.708124 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:03.023231 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:03.068078 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:03.720918 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:03.926145 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:04.041849 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:04.075447 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:04.906335 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:05.139617 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:05.161987 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:05.431347 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:06.037162 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:06.087607 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:06.659204 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:06.808274 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:06.865266 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:07.180481 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:07.266597 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:07.302448 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:08.180300 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:08.339319 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:08.398443 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:08.532454 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:08.925184 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:08.951998 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:09.209924 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:09.235169 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:09.326915 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:09.775735 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:09.854163 [ 7168 ] {60e682f5-2b4e-4ce1-993c-448191d7d2bf} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55004) (comment: 00653_verification_monotonic_data_load.sh) (in query: CREATE TABLE unsigned_integer_test_table (val UInt32) ENGINE = MergeTree ORDER BY val SETTINGS index_granularity = 1, index_granularity_bytes = 0, min_bytes_for_wide_part = 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c45d4f6 8. ./contrib/llvm-project/libcxx/include/optional:260: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>) @ 0x000000002cbacf28 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1460: std::shared_ptr std::allocate_shared[abi:v15007], DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>, void>(std::allocator const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>&&) @ 0x000000002cbabafa 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::create(DB::StorageFactory::Arguments const&) @ 0x000000002cba21b8 11. ./build_docker/./src/Storages/StorageFactory.cpp:225: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x000000002b77bf53 12. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1659: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291b9254 13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:09.870057 [ 7168 ] {60e682f5-2b4e-4ce1-993c-448191d7d2bf} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c45d4f6 8. ./contrib/llvm-project/libcxx/include/optional:260: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>) @ 0x000000002cbacf28 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1460: std::shared_ptr std::allocate_shared[abi:v15007], DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>, void>(std::allocator const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>&&) @ 0x000000002cbabafa 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::create(DB::StorageFactory::Arguments const&) @ 0x000000002cba21b8 11. ./build_docker/./src/Storages/StorageFactory.cpp:225: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x000000002b77bf53 12. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1659: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291b9254 13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:10.065110 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:10.093296 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:10.339268 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:10.543937 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:10.644330 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:10.686664 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:11.616740 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:11.892063 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:11.949008 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:12.510617 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:12.867485 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:13.000037 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:13.090568 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:13.190518 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:14.062754 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:14.073121 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:14.334653 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:14.460941 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:15.361380 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:15.404184 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:15.569663 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:15.741167 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:15.829621 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:16.207736 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:17.001068 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:17.081979 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:17.088704 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:17.690777 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:17.733315 [ 7168 ] {937750c0-44e0-422d-bcc0-df6a7111c956} executeQuery: Code: 60. DB::Exception: Table test_y1zzz7ev.unsigned_integer_test_table does not exist. Maybe you meant test_y1zzz7ev.signed_integer_test_table?. (UNKNOWN_TABLE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48348) (comment: 00653_verification_monotonic_data_load.sh) (in query: INSERT INTO unsigned_integer_test_table VALUES ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&, String&&) @ 0x000000000ac6a006 4. ./build_docker/./src/Databases/IDatabase.cpp:39: DB::IDatabase::getTable(String const&, std::shared_ptr) const @ 0x0000000027dd1c6a 5. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr, std::optional*) const @ 0x00000000286af00b 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:801: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr) const @ 0x00000000286bb117 7. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:0: DB::InterpreterInsertQuery::getTable(DB::ASTInsertQuery&) @ 0x000000002934b836 8. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:748: DB::InterpreterInsertQuery::execute() @ 0x000000002935ad49 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:17.739815 [ 7168 ] {937750c0-44e0-422d-bcc0-df6a7111c956} TCPHandler: Code: 60. DB::Exception: Table test_y1zzz7ev.unsigned_integer_test_table does not exist. Maybe you meant test_y1zzz7ev.signed_integer_test_table?. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&, String&&) @ 0x000000000ac6a006 4. ./build_docker/./src/Databases/IDatabase.cpp:39: DB::IDatabase::getTable(String const&, std::shared_ptr) const @ 0x0000000027dd1c6a 5. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr, std::optional*) const @ 0x00000000286af00b 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:801: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr) const @ 0x00000000286bb117 7. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:0: DB::InterpreterInsertQuery::getTable(DB::ASTInsertQuery&) @ 0x000000002934b836 8. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:748: DB::InterpreterInsertQuery::execute() @ 0x000000002935ad49 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:18.166227 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:18.201342 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:18.862431 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:19.429968 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:19.490819 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:19.523803 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:19.954905 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:20.031835 [ 3355 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {f2eeeeb5-70de-4a33-835e-3bff489e4be8::202509_1_54_11}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:20.123537 [ 7168 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::1]:48376, local: [::1]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:20.664565 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:20.754460 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:20.766859 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:21.183550 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:21.307085 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:21.969416 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:22.001741 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:22.248593 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:22.657626 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:22.800726 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:22.890356 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:22.947096 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:23.306263 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:23.912469 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:24.006414 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:24.182800 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:24.663909 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:24.740940 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:24.764431 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:25.023777 [ 7168 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::1]:48424, local: [::1]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:25.360756 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:25.448847 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:25.624130 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:25.698427 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:25.754468 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:26.176053 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:26.226717 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:26.591471 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:26.744944 [ 7168 ] {ae6108de-66c1-45a1-9cd1-62d2fe81d2dc} executeQuery: Code: 60. DB::Exception: Unknown table expression identifier 'unsigned_integer_test_table' in scope SELECT count() FROM unsigned_integer_test_table WHERE toInt64(val) = 0. (UNKNOWN_TABLE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48444) (comment: 00653_verification_monotonic_data_load.sh) (in query: SELECT count() FROM unsigned_integer_test_table WHERE toInt64(val) == 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:26.754921 [ 7168 ] {ae6108de-66c1-45a1-9cd1-62d2fe81d2dc} TCPHandler: Code: 60. DB::Exception: Unknown table expression identifier 'unsigned_integer_test_table' in scope SELECT count() FROM unsigned_integer_test_table WHERE toInt64(val) = 0. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:26.805246 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:27.184326 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:27.208603 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:27.384720 [ 7168 ] {3ec4d371-23c4-443c-be8c-a358c90cfd3c} executeQuery: Code: 60. DB::Exception: Unknown table expression identifier 'unsigned_integer_test_table' in scope SELECT count() FROM unsigned_integer_test_table WHERE toUInt64(val) = 0. (UNKNOWN_TABLE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50204) (comment: 00653_verification_monotonic_data_load.sh) (in query: SELECT count() FROM unsigned_integer_test_table WHERE toUInt64(val) == 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:27.415506 [ 7168 ] {3ec4d371-23c4-443c-be8c-a358c90cfd3c} TCPHandler: Code: 60. DB::Exception: Unknown table expression identifier 'unsigned_integer_test_table' in scope SELECT count() FROM unsigned_integer_test_table WHERE toUInt64(val) = 0. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:27.607550 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:27.853988 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:28.003562 [ 7168 ] {744cc685-1a51-4d09-a196-b9a78a326669} executeQuery: Code: 60. DB::Exception: Unknown table expression identifier 'unsigned_integer_test_table' in scope SELECT count() FROM unsigned_integer_test_table WHERE toInt32(val) = 0. (UNKNOWN_TABLE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50214) (comment: 00653_verification_monotonic_data_load.sh) (in query: SELECT count() FROM unsigned_integer_test_table WHERE toInt32(val) == 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:28.010486 [ 7168 ] {744cc685-1a51-4d09-a196-b9a78a326669} TCPHandler: Code: 60. DB::Exception: Unknown table expression identifier 'unsigned_integer_test_table' in scope SELECT count() FROM unsigned_integer_test_table WHERE toInt32(val) = 0. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:28.301437 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:28.396016 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:28.570041 [ 7168 ] {81448bc9-469c-4130-b99d-b59f9874a358} executeQuery: Code: 60. DB::Exception: Unknown table expression identifier 'unsigned_integer_test_table' in scope SELECT count() FROM unsigned_integer_test_table WHERE toUInt32(val) = 0. (UNKNOWN_TABLE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50216) (comment: 00653_verification_monotonic_data_load.sh) (in query: SELECT count() FROM unsigned_integer_test_table WHERE toUInt32(val) == 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:28.580956 [ 7168 ] {81448bc9-469c-4130-b99d-b59f9874a358} TCPHandler: Code: 60. DB::Exception: Unknown table expression identifier 'unsigned_integer_test_table' in scope SELECT count() FROM unsigned_integer_test_table WHERE toUInt32(val) = 0. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:28.603698 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:29.011281 [ 7168 ] {952f6383-abef-4813-b868-3afbab6d2960} executeQuery: Code: 60. DB::Exception: Unknown table expression identifier 'unsigned_integer_test_table' in scope SELECT count() FROM unsigned_integer_test_table WHERE toInt16(val) = 0. (UNKNOWN_TABLE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50230) (comment: 00653_verification_monotonic_data_load.sh) (in query: SELECT count() FROM unsigned_integer_test_table WHERE toInt16(val) == 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:29.018142 [ 7168 ] {952f6383-abef-4813-b868-3afbab6d2960} TCPHandler: Code: 60. DB::Exception: Unknown table expression identifier 'unsigned_integer_test_table' in scope SELECT count() FROM unsigned_integer_test_table WHERE toInt16(val) = 0. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:29.335818 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:29.350479 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:29.388860 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:29.569149 [ 7168 ] {ae6f2c86-913f-4df0-aec3-a9ded7baeaaa} executeQuery: Code: 60. DB::Exception: Unknown table expression identifier 'unsigned_integer_test_table' in scope SELECT count() FROM unsigned_integer_test_table WHERE toUInt16(val) = 0. (UNKNOWN_TABLE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50236) (comment: 00653_verification_monotonic_data_load.sh) (in query: SELECT count() FROM unsigned_integer_test_table WHERE toUInt16(val) == 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:29.581540 [ 7168 ] {ae6f2c86-913f-4df0-aec3-a9ded7baeaaa} TCPHandler: Code: 60. DB::Exception: Unknown table expression identifier 'unsigned_integer_test_table' in scope SELECT count() FROM unsigned_integer_test_table WHERE toUInt16(val) = 0. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:29.682988 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:29.711969 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:30.056011 [ 7168 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::1]:50250, local: [::1]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:30.169705 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:30.452932 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:30.458883 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:30.474937 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:30.564617 [ 7168 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::1]:50254, local: [::1]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:30.752950 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:30.864427 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:30.886385 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:30.983330 [ 7168 ] {31461099-2d65-4d8e-beb7-d633706e643c} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=728). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50262) (comment: 00653_verification_monotonic_data_load.sh) (in query: SELECT count() FROM enum_test_table WHERE toInt16(val) == 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:30.986950 [ 7168 ] {31461099-2d65-4d8e-beb7-d633706e643c} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=728). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:31.696937 [ 7168 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::1]:50264, local: [::1]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:31.741105 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:32.050325 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:32.067801 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:32.192722 [ 7168 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::1]:50280, local: [::1]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:32.499690 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:32.687050 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:32.705255 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:33.292384 [ 7168 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::1]:50308, local: [::1]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:33.497368 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:33.629216 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:33.665484 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:34.025724 [ 7168 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::1]:50318, local: [::1]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:34.162379 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:34.213397 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:34.392317 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:34.715935 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:34.742800 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:34.771978 [ 7168 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::1]:50320, local: [::1]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:34.778140 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:35.389309 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:35.417677 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:35.488009 [ 7168 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::1]:50330, local: [::1]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:35.726283 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:36.702882 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:36.765837 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:36.778635 [ 7168 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::1]:50352, local: [::1]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:36.785704 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:37.244118 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:37.267267 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:37.358575 [ 2766 ] {} DatabaseCatalog: Cannot drop tables. Will retry later.: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000286c6b9c 8. ./contrib/llvm-project/libcxx/include/vector:590: DB::DatabaseCatalog::dropTableDataTask() @ 0x00000000286c7466 9. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 11. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 14. asan_thread_start(void*) @ 0x000000000aa43059 15. ? @ 0x00007f12cd75eac3 16. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:37.554253 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:37.555267 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:37.603384 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:37.990332 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:38.595884 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:38.632126 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:38.832984 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:38.952699 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:38.988299 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:39.768579 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:39.797463 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:39.868051 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:40.859034 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:41.047423 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:41.077710 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:41.187074 [ 3682 ] {7b4d45bb-8c70-4c21-bf0f-7c24c8e21855} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46148) (comment: 00653_verification_monotonic_data_load.sh) (in query: DROP DATABASE IF EXISTS test_y1zzz7ev ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:41.190500 [ 3682 ] {7b4d45bb-8c70-4c21-bf0f-7c24c8e21855} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:41.644505 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:41.645609 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:41.674844 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:41.800328 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:42.293536 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:42.610305 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:42.671231 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:42.820228 [ 3682 ] {3e5292a2-52c5-4fa5-8ef6-d37d5bec6378} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46180) (comment: 03289_explain_syntax_statistics.sql) (in query: DROP DATABASE IF EXISTS test_7pba9tgc ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:42.824164 [ 3682 ] {3e5292a2-52c5-4fa5-8ef6-d37d5bec6378} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:42.932208 [ 7168 ] {a9832dd0-aab3-48b5-a17f-92c2e9cad287} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46184) (comment: 03289_explain_syntax_statistics.sql) (in query: DROP DATABASE IF EXISTS test_7pba9tgc ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:42.934631 [ 7168 ] {a9832dd0-aab3-48b5-a17f-92c2e9cad287} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:42.994238 [ 3682 ] {e81ad54f-7517-4911-8342-925699e6633e} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46188) (comment: 03289_explain_syntax_statistics.sql) (in query: DROP DATABASE IF EXISTS test_7pba9tgc ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:42.996190 [ 3682 ] {e81ad54f-7517-4911-8342-925699e6633e} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:43.053459 [ 7168 ] {7c0da159-3359-4a6a-b465-933974db4ce6} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46196) (comment: 03289_explain_syntax_statistics.sql) (in query: DROP DATABASE IF EXISTS test_7pba9tgc ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:43.056641 [ 7168 ] {7c0da159-3359-4a6a-b465-933974db4ce6} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:43.370586 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:43.383636 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:43.394422 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:43.728454 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:43.730708 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:43.748031 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:44.243776 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:44.258203 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:44.419717 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:45.296761 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:45.297708 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:45.399678 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:46.175643 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:46.379255 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:46.405039 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:46.829379 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:47.056397 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:47.073565 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:47.481827 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:47.504903 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:47.551837 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:47.898887 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:48.285335 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:48.349347 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:48.453020 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:48.670553 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:48.681356 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:49.026598 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:49.045937 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:49.225357 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:49.665190 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:49.679884 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:49.810603 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:50.473478 [ 3682 ] {b5f42f16-995a-4a83-975e-c69c7e33ef6c} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43582) (comment: 00688_low_cardinality_dictionary_deserialization.sql) (in query: select sum(toUInt64(str)), sum(toUInt64(pat)) from lc_dict_reading where val < 8129 or val > 8192 * 4;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:50.479654 [ 3682 ] {b5f42f16-995a-4a83-975e-c69c7e33ef6c} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:50.504094 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:50.718720 [ 7168 ] {3eb2831f-f474-4531-a331-988e61da16c3} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:44204) (comment: 00688_low_cardinality_dictionary_deserialization.sql) (in query: DROP DATABASE IF EXISTS test_qkcr8c6r ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:15:50.722913 [ 7168 ] {3eb2831f-f474-4531-a331-988e61da16c3} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:50.858508 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:50.872429 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:51.140178 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:51.177963 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:51.235048 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:51.570017 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:51.663851 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:52.120891 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:52.192375 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:52.234907 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:52.708396 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:52.976684 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:52.992624 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:53.495388 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:53.557297 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:53.689308 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:54.040061 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:54.665659 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:54.689351 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:54.878223 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:55.111276 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:55.172727 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:55.188696 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:55.787324 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:55.839056 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:56.071644 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:56.103710 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:56.144782 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:56.539577 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:56.747005 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:56.782627 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:57.042461 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:57.524974 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:57.547776 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:58.146038 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:58.510270 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:58.534637 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:58.927372 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:59.138734 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:59.312292 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:59.649187 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:15:59.680626 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:00.077305 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:00.416155 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:00.428708 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:00.870077 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:01.329596 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:01.350499 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:01.911070 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:02.165706 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:02.187844 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:02.260148 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:02.703280 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:03.330912 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:03.452662 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:03.607612 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:03.715447 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:04.321549 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:04.335139 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:04.702361 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:04.855827 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:04.876518 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:05.489326 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:06.056420 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:06.075153 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:06.145531 [ 3682 ] {b49b5258-84ba-4e92-8342-434399df8807} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:41838) (comment: 02354_read_in_order_prewhere.sql) (in query: SELECT Num FROM order WHERE Type = 1 AND ID = 1 ORDER BY Num ASC limit 5;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:06.159267 [ 3682 ] {b49b5258-84ba-4e92-8342-434399df8807} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:06.419802 [ 9743 ] {0f70102a-7ba4-4255-9df9-1cd1e93019f7} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:49702) (comment: 02354_read_in_order_prewhere.sql) (in query: DROP DATABASE IF EXISTS test_vion1eex ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:06.424666 [ 9743 ] {0f70102a-7ba4-4255-9df9-1cd1e93019f7} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:06.438489 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:06.583567 [ 5079 ] {} test_vion1eex.order (733e15c4-7f5f-42b7-af4d-b6a2e74817bb): dropAllData: got exception removing parts from disk, removing successfully removed parts from memory. 2025.09.25 21:16:06.584814 [ 5079 ] {} DatabaseCatalog: Cannot drop table test_vion1eex.order (733e15c4-7f5f-42b7-af4d-b6a2e74817bb). Will retry later.: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c47c445 8. ./contrib/llvm-project/libcxx/include/vector:547: DB::MergeTreeData::clearPartsFromFilesystem(std::vector, std::allocator>> const&, bool, std::unordered_set, std::equal_to, std::allocator>*) @ 0x000000002c47a78e 9. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::dropAllData() @ 0x000000002c4862d9 10. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::dropTableFinally(DB::DatabaseCatalog::TableMarkedAsDropped const&) @ 0x00000000286c7c77 11. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: void std::__function::__policy_invoker::__call_impl, std::allocator>>)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x00000000286d9744 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x00000000263a3ea5 13. ./contrib/llvm-project/libcxx/include/future:0: std::packaged_task::operator()() @ 0x000000001bba33ac 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:07.088029 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:07.102870 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:07.611560 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:08.040086 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:08.077974 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:08.435254 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:09.010815 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:09.039960 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:09.491046 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:09.603401 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:09.616014 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:09.913347 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:10.238431 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:10.261861 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:10.488021 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:11.070982 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:11.476147 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:11.575467 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:11.662192 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:11.778807 [ 5084 ] {} test_vion1eex.order (733e15c4-7f5f-42b7-af4d-b6a2e74817bb): dropAllData: got exception removing parts from disk, removing successfully removed parts from memory. 2025.09.25 21:16:11.840306 [ 5084 ] {} DatabaseCatalog: Cannot drop table test_vion1eex.order (733e15c4-7f5f-42b7-af4d-b6a2e74817bb). Will retry later.: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c47c445 8. ./contrib/llvm-project/libcxx/include/vector:547: DB::MergeTreeData::clearPartsFromFilesystem(std::vector, std::allocator>> const&, bool, std::unordered_set, std::equal_to, std::allocator>*) @ 0x000000002c47a78e 9. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::dropAllData() @ 0x000000002c4862d9 10. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::dropTableFinally(DB::DatabaseCatalog::TableMarkedAsDropped const&) @ 0x00000000286c7c77 11. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: void std::__function::__policy_invoker::__call_impl, std::allocator>>)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x00000000286d9744 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x00000000263a3ea5 13. ./contrib/llvm-project/libcxx/include/future:0: std::packaged_task::operator()() @ 0x000000001bba33ac 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:12.415887 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:12.448095 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:12.449374 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:12.823739 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:12.848185 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:13.448973 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:13.767376 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:13.781784 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:14.103879 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:14.105171 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:14.121349 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:14.841889 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:14.925447 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:14.942792 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:15.526551 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:15.542847 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:15.984637 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:16.022323 [ 3361 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {df97ab97-a5cf-4ea8-bd10-026470f5fb5c::202509_1_86_17}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:16.599363 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:16.772010 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:16.800029 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:17.067842 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:17.189519 [ 3794 ] {} test_vion1eex.order (733e15c4-7f5f-42b7-af4d-b6a2e74817bb): dropAllData: got exception removing parts from disk, removing successfully removed parts from memory. 2025.09.25 21:16:17.198203 [ 3794 ] {} DatabaseCatalog: Cannot drop table test_vion1eex.order (733e15c4-7f5f-42b7-af4d-b6a2e74817bb). Will retry later.: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c47c445 8. ./contrib/llvm-project/libcxx/include/vector:547: DB::MergeTreeData::clearPartsFromFilesystem(std::vector, std::allocator>> const&, bool, std::unordered_set, std::equal_to, std::allocator>*) @ 0x000000002c47a78e 9. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::dropAllData() @ 0x000000002c4862d9 10. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::dropTableFinally(DB::DatabaseCatalog::TableMarkedAsDropped const&) @ 0x00000000286c7c77 11. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: void std::__function::__policy_invoker::__call_impl, std::allocator>>)::$_0, void ()>>(std::__function::__policy_storage const*) @ 0x00000000286d9744 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x00000000263a3ea5 13. ./contrib/llvm-project/libcxx/include/future:0: std::packaged_task::operator()() @ 0x000000001bba33ac 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:17.212208 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:17.234382 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:17.495137 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:18.297045 [ 9743 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:18.396425 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:18.407326 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:18.466833 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:18.973001 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:19.159765 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:19.568263 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:19.587115 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:19.863971 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:19.886566 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:20.242471 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:20.423694 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:20.442364 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:20.916153 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:20.969040 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:20.984276 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:21.629982 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:21.639673 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:21.728111 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:22.321192 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:22.335067 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:22.619226 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:23.055164 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:23.090763 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:23.215563 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:23.864591 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:23.869446 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:23.892075 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:24.586407 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:24.636695 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:24.785111 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:25.521130 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:25.565497 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:25.641455 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:26.256895 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:26.367087 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:26.414256 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:26.482724 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:26.634331 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:27.210016 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:27.265142 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:27.305750 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:28.037729 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:28.066265 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:28.144855 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:28.577437 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:29.353779 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:29.406335 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:29.413508 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:29.751278 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:30.280312 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:30.494777 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:30.628335 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:30.654555 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:31.282158 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:31.445739 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:31.476090 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:31.483998 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:32.046086 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:32.064348 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:32.486008 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:32.870813 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:33.016172 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:33.064289 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:33.531023 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:33.934650 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:33.988087 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:34.271619 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:35.153855 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:35.178390 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:35.202013 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:35.784137 [ 3682 ] {434a3489-81de-4b67-bd7c-4b886af9a5f3} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_6nj3uy8j, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46450) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from database rdb_test_6nj3uy8j), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./src/Databases/DatabaseReplicated.h:76: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab2fb 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:35.797347 [ 3682 ] {434a3489-81de-4b67-bd7c-4b886af9a5f3} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_6nj3uy8j, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./src/Databases/DatabaseReplicated.h:76: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab2fb 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:36.147179 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:36.191619 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:36.286406 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:36.539430 [ 3682 ] {7ff7f33f-fb54-466d-a105-59ed50c82134} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_6nj3uy8j, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46462) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 'r1' from shard 's1' from database rdb_test_6nj3uy8j), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./src/Databases/DatabaseReplicated.h:76: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab2fb 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:36.554300 [ 3682 ] {7ff7f33f-fb54-466d-a105-59ed50c82134} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_6nj3uy8j, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./src/Databases/DatabaseReplicated.h:76: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab2fb 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:36.632683 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:37.215831 [ 3682 ] {8f088d3a-7e16-4ddc-b1f4-41e0f61c1d1b} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_6nj3uy8j, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43946) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from zkpath '/test/test_6nj3uy8j/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:37.223704 [ 3682 ] {8f088d3a-7e16-4ddc-b1f4-41e0f61c1d1b} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_6nj3uy8j, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:37.347531 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:37.388406 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:37.548985 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:37.778768 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:37.855802 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:38.042255 [ 3682 ] {162ddeef-d0f4-4ca7-a21e-90503db95355} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_6nj3uy8j, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43956) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from zkpath '/test/test_6nj3uy8j/rdb/'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:38.064945 [ 3682 ] {162ddeef-d0f4-4ca7-a21e-90503db95355} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_6nj3uy8j, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:38.644908 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:38.911909 [ 3682 ] {04bd8752-f9b8-4c40-a370-a04fd54f8f44} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_6nj3uy8j, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43964) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 'r1' from shard 's1' from zkpath '/test/test_6nj3uy8j/rdb/'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:38.921605 [ 3682 ] {04bd8752-f9b8-4c40-a370-a04fd54f8f44} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_6nj3uy8j, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:39.052768 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:39.096308 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:39.115016 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:39.824891 [ 3682 ] {48cd9f2b-934e-4915-bf3a-92af765f8ed3} executeQuery: Code: 36. DB::Exception: Path /test/test_6nj3uy8j does not look like a path of Replicated database. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43974) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from zkpath '/test/test_6nj3uy8j/'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1501: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cbfc 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:39.844120 [ 3682 ] {48cd9f2b-934e-4915-bf3a-92af765f8ed3} TCPHandler: Code: 36. DB::Exception: Path /test/test_6nj3uy8j does not look like a path of Replicated database. (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1501: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cbfc 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:39.995074 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:40.323745 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:40.362071 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:40.568648 [ 3682 ] {d2262ceb-a876-4098-b7e3-336e8056829f} executeQuery: Code: 36. DB::Exception: Replica s2|r1 does not exist (database path: /test/test_6nj3uy8j/rdb). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43978) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's2|r1' from zkpath '/test/test_6nj3uy8j/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:40.640418 [ 3682 ] {d2262ceb-a876-4098-b7e3-336e8056829f} TCPHandler: Code: 36. DB::Exception: Replica s2|r1 does not exist (database path: /test/test_6nj3uy8j/rdb). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:41.064629 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:41.156216 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:41.185174 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:41.405026 [ 3682 ] {a24915d6-192c-4867-86c5-07c3350b476d} executeQuery: Code: 36. DB::Exception: Replica r1|s1 does not exist (database path: /test/test_6nj3uy8j/rdb). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43980) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1' from shard 'r1' from zkpath '/test/test_6nj3uy8j/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:41.410579 [ 3682 ] {a24915d6-192c-4867-86c5-07c3350b476d} TCPHandler: Code: 36. DB::Exception: Replica r1|s1 does not exist (database path: /test/test_6nj3uy8j/rdb). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:41.837494 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:42.009586 [ 3682 ] {9c05902c-035a-421a-955e-a01590c04815} executeQuery: Code: 36. DB::Exception: Replica s1|s1|r1 does not exist (database path: /test/test_6nj3uy8j/rdb). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43986) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from shard 's1' from zkpath '/test/test_6nj3uy8j/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:42.022403 [ 3682 ] {9c05902c-035a-421a-955e-a01590c04815} TCPHandler: Code: 36. DB::Exception: Replica s1|s1|r1 does not exist (database path: /test/test_6nj3uy8j/rdb). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:42.025687 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:42.037571 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:42.055831 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:42.638051 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:42.670737 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:42.923695 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:43.013263 [ 3682 ] {4299dfe5-3884-4204-9ee4-7704b310b441} executeQuery: Code: 36. DB::Exception: Invalid replica name, '/' is not allowed: s2/r1. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43994) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's2/r1' from zkpath '/test/test_6nj3uy8j/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x000000000aafc574 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1492: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cca3 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:43.039672 [ 3682 ] {4299dfe5-3884-4204-9ee4-7704b310b441} TCPHandler: Code: 36. DB::Exception: Invalid replica name, '/' is not allowed: s2/r1. (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x000000000aafc574 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1492: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cca3 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:43.526697 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:43.553720 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:43.867119 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:44.172990 [ 5082 ] {} DDLWorker(rdb_test_6nj3uy8j_2): Error on initialization of rdb_test_6nj3uy8j_2: Code: 57. DB::Exception: Directory for table data store/0ac/0ac24fa8-546f-4669-a103-27b75a4faf8c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:44.189022 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:44.423145 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:44.490900 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:44.795631 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:44.849356 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:44.910995 [ 3794 ] {} DDLWorker(rdb_test_6nj3uy8j_3): Error on initialization of rdb_test_6nj3uy8j_3: Code: 57. DB::Exception: Directory for table data store/0ac/0ac24fa8-546f-4669-a103-27b75a4faf8c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:45.047657 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:45.692355 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:45.706884 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:45.727148 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:46.582911 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:46.931468 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:47.050410 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:47.110946 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:47.135119 [ 3682 ] {98411d97-f1f6-4b1c-bdc6-b3a7dfcafe72} executeQuery: Code: 36. DB::Exception: Replica s1|r1 is active, cannot drop it (database path: /test/test_6nj3uy8j/rdb). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48420) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from database rdb_test_6nj3uy8j_2), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1513: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cde0 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab31e 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:47.140646 [ 3682 ] {98411d97-f1f6-4b1c-bdc6-b3a7dfcafe72} TCPHandler: Code: 36. DB::Exception: Replica s1|r1 is active, cannot drop it (database path: /test/test_6nj3uy8j/rdb). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1513: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cde0 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab31e 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:16:47.545640 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:47.976561 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:48.004903 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:48.191006 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:48.928721 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:48.949569 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:49.117790 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:49.260405 [ 5082 ] {} DDLWorker(rdb_test_6nj3uy8j_2): Error on initialization of rdb_test_6nj3uy8j_2: Code: 57. DB::Exception: Directory for table data store/c89/c89e28b0-9f0a-4c5f-a8c2-b066a329d26c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:49.737207 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:49.771265 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:49.864241 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:49.985010 [ 3794 ] {} DDLWorker(rdb_test_6nj3uy8j_3): Error on initialization of rdb_test_6nj3uy8j_3: Code: 57. DB::Exception: Directory for table data store/8e4/8e432cf0-bdde-4ec4-a4f3-93eab9ef2aad/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:50.215877 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:50.489519 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:50.549336 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:50.784152 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:51.371293 [ 5065 ] {} DDLWorker(rdb_test_6nj3uy8j_9): Error on initialization of rdb_test_6nj3uy8j_9: Code: 57. DB::Exception: Directory for table data store/8e4/8e432cf0-bdde-4ec4-a4f3-93eab9ef2aad/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:51.622581 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:51.649083 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:51.700203 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:51.990447 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:52.763766 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:52.795577 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:52.804661 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:53.817212 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:53.948487 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:54.040145 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:54.311477 [ 5082 ] {} DDLWorker(rdb_test_6nj3uy8j_2): Error on initialization of rdb_test_6nj3uy8j_2: Code: 57. DB::Exception: Directory for table data store/fcf/fcfb9249-343b-4de9-a3ba-0cafe995637d/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:54.709731 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:54.721988 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:54.931448 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:55.037898 [ 3794 ] {} DDLWorker(rdb_test_6nj3uy8j_3): Error on initialization of rdb_test_6nj3uy8j_3: Code: 57. DB::Exception: Directory for table data store/fcf/fcfb9249-343b-4de9-a3ba-0cafe995637d/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:55.062663 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:55.072854 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:55.504789 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:55.517667 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:55.821479 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:56.408917 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:56.499934 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:56.523224 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:56.645695 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:57.152365 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:57.649383 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:57.706602 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:57.925360 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:58.448619 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:58.463935 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:58.791511 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:58.922746 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:58.937615 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:59.525896 [ 5082 ] {} DDLWorker(rdb_test_6nj3uy8j_2): Error on initialization of rdb_test_6nj3uy8j_2: Code: 57. DB::Exception: Directory for table data store/fcf/fcfb9249-343b-4de9-a3ba-0cafe995637d/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:59.548675 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:59.747587 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:59.852121 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:16:59.936921 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:00.117097 [ 3794 ] {} DDLWorker(rdb_test_6nj3uy8j_3): Error on initialization of rdb_test_6nj3uy8j_3: Code: 57. DB::Exception: Directory for table data store/fcf/fcfb9249-343b-4de9-a3ba-0cafe995637d/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:00.618330 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:00.646170 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:00.713329 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:01.273117 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:01.291899 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:01.650954 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:01.680610 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:01.864996 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:02.321921 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:02.475563 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:02.495626 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:02.777640 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:02.791822 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:03.037578 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:03.683228 [ 3682 ] {1835e9d3-a4d1-4e30-ae1e-5f7f02942e97} executeQuery: Code: 159. DB::Exception: Distributed DDL task /test/test_6nj3uy8j/rdb/log/query-0000000009 is not finished on 3 of 4 hosts (0 of them are currently executing the task, 2 are inactive). They are going to execute the query in background. Was waiting for 5.247955569 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55076) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_6nj3uy8j.t22 (n int) engine=Log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::DDLQueryStatusSource::generate() @ 0x0000000029b11829 4. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 5. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 10. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d3463e7 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:03.693261 [ 3682 ] {1835e9d3-a4d1-4e30-ae1e-5f7f02942e97} TCPHandler: Code: 159. DB::Exception: Distributed DDL task /test/test_6nj3uy8j/rdb/log/query-0000000009 is not finished on 3 of 4 hosts (0 of them are currently executing the task, 2 are inactive). They are going to execute the query in background. Was waiting for 5.247955569 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::DDLQueryStatusSource::generate() @ 0x0000000029b11829 4. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 5. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 10. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d3463e7 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:04.004198 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:04.039262 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:04.100966 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:04.724348 [ 5082 ] {} DDLWorker(rdb_test_6nj3uy8j_2): Error on initialization of rdb_test_6nj3uy8j_2: Code: 57. DB::Exception: Directory for table data store/fcf/fcfb9249-343b-4de9-a3ba-0cafe995637d/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:04.820535 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:04.830660 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:04.957338 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:05.173338 [ 3794 ] {} DDLWorker(rdb_test_6nj3uy8j_3): Error on initialization of rdb_test_6nj3uy8j_3: Code: 57. DB::Exception: Directory for table data store/c89/c89e28b0-9f0a-4c5f-a8c2-b066a329d26c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:05.484862 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:05.722311 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:05.750036 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:06.629651 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:06.768785 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:06.784893 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:07.039224 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:07.223257 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:07.336916 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:07.738449 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:07.782580 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:07.838905 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:08.591176 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:08.609093 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:08.917695 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:09.336729 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:09.446941 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:09.659865 [ 3682 ] {c90249ad-2046-4937-9148-2f3655065988} executeQuery: Code: 159. DB::Exception: Distributed DDL task /test/test_6nj3uy8j/rdb/log/query-0000000010 is not finished on 3 of 4 hosts (0 of them are currently executing the task, 2 are inactive). They are going to execute the query in background. Was waiting for 5.047957274 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55078) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_6nj3uy8j.t33 (n int) engine=Log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::DDLQueryStatusSource::generate() @ 0x0000000029b11829 4. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 5. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 10. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:09.664243 [ 3682 ] {c90249ad-2046-4937-9148-2f3655065988} TCPHandler: Code: 159. DB::Exception: Distributed DDL task /test/test_6nj3uy8j/rdb/log/query-0000000010 is not finished on 3 of 4 hosts (0 of them are currently executing the task, 2 are inactive). They are going to execute the query in background. Was waiting for 5.047957274 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::DDLQueryStatusSource::generate() @ 0x0000000029b11829 4. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 5. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 10. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:09.836414 [ 5082 ] {} DDLWorker(rdb_test_6nj3uy8j_2): Error on initialization of rdb_test_6nj3uy8j_2: Code: 57. DB::Exception: Directory for table data store/fcf/fcfb9249-343b-4de9-a3ba-0cafe995637d/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:09.922967 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:09.939856 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:09.998552 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:10.254129 [ 3794 ] {} DDLWorker(rdb_test_6nj3uy8j_3): Error on initialization of rdb_test_6nj3uy8j_3: Code: 57. DB::Exception: Directory for table data store/c89/c89e28b0-9f0a-4c5f-a8c2-b066a329d26c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:10.616286 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:10.683655 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:10.707123 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:11.251435 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:11.369216 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:11.383783 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:11.474314 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:11.651577 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:11.821613 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:12.047324 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:12.429595 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:12.451260 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:12.599414 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:13.082687 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:13.543681 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:13.568478 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:13.765669 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:14.099844 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:14.392768 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:14.408353 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:14.744664 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:14.764824 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:14.992816 [ 5082 ] {} DDLWorker(rdb_test_6nj3uy8j_2): Error on initialization of rdb_test_6nj3uy8j_2: Code: 57. DB::Exception: Directory for table data store/cde/cde199a0-1a77-40d1-b5ef-f7d84154bd7e/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:14.993495 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:15.406701 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:15.467699 [ 3794 ] {} DDLWorker(rdb_test_6nj3uy8j_3): Error on initialization of rdb_test_6nj3uy8j_3: Code: 57. DB::Exception: Directory for table data store/8e4/8e432cf0-bdde-4ec4-a4f3-93eab9ef2aad/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:15.768690 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:15.813861 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:15.820321 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:16.381822 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:16.404187 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:16.516660 [ 3785 ] {} DDLWorker(rdb_test_6nj3uy8j_9): Error on initialization of rdb_test_6nj3uy8j_9: Code: 57. DB::Exception: Directory for table data store/fcf/fcfb9249-343b-4de9-a3ba-0cafe995637d/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:16.568715 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:16.839422 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:16.872470 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:17.395666 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:17.716632 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:17.732523 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:18.522432 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:18.627357 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:18.651586 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:18.692141 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:19.362549 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:19.377831 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:19.482748 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:20.060293 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:20.087866 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:20.224782 [ 5082 ] {} DDLWorker(rdb_test_6nj3uy8j_2): Error on initialization of rdb_test_6nj3uy8j_2: Code: 57. DB::Exception: Directory for table data store/fcf/fcfb9249-343b-4de9-a3ba-0cafe995637d/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:20.442347 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:20.459918 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:20.522219 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:20.617301 [ 3794 ] {} DDLWorker(rdb_test_6nj3uy8j_3): Error on initialization of rdb_test_6nj3uy8j_3: Code: 57. DB::Exception: Directory for table data store/cde/cde199a0-1a77-40d1-b5ef-f7d84154bd7e/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:21.050168 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:21.191550 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:21.201260 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:21.544556 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:21.561525 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:21.775977 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:21.917278 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:22.234515 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:22.280718 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:22.524416 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:23.229607 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:23.462060 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:23.685105 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:23.742240 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:23.952296 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:24.975666 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:24.990203 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:25.027521 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:25.363836 [ 5082 ] {} DDLWorker(rdb_test_6nj3uy8j_2): Error on initialization of rdb_test_6nj3uy8j_2: Code: 57. DB::Exception: Directory for table data store/cde/cde199a0-1a77-40d1-b5ef-f7d84154bd7e/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:25.389861 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:25.415403 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:25.961509 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:26.181849 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:26.202014 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:26.926004 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:26.966793 [ 3682 ] {8aa7962e-4433-4c87-9526-2105c8a679e2} DatabaseReplicated (rdb_test_6nj3uy8j_3): Database rdb_test_6nj3uy8j_3 exists locally, but its replica does not exist in ZooKeeper (/test/test_6nj3uy8j/rdb/replicas/s2|r1). Assuming it was dropped, will not try to start it up 2025.09.25 21:17:27.246213 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:27.353942 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:27.853310 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:27.908462 [ 3682 ] {cf220bc5-956a-4f23-8d60-0e1f315653de} executeQuery: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60236) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_6nj3uy8j_3.t2 as system.query_log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:27.929132 [ 3682 ] {cf220bc5-956a-4f23-8d60-0e1f315653de} TCPHandler: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:28.260218 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:28.306203 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:28.339475 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:28.951028 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:29.038043 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:29.052561 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:29.873088 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:29.885329 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:30.038855 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:30.599463 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:30.662393 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:30.754962 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:31.239876 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:31.854758 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:31.904202 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:32.165641 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:32.617510 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:32.852704 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:32.876479 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:32.905783 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:33.130709 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:33.359537 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:33.554352 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:33.577985 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:33.720089 [ 3682 ] {ca6acad4-4fba-41e0-99ad-df1561ff67dd} DatabaseReplicated (rdb_test_6nj3uy8j_2): Database rdb_test_6nj3uy8j_2 exists locally, but its replica does not exist in ZooKeeper (/test/test_6nj3uy8j/rdb/replicas/s1|r2). Assuming it was dropped, will not try to start it up 2025.09.25 21:17:34.028431 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:34.041614 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:34.401409 [ 3682 ] {245795f0-2929-4f16-84a8-4f69cdfc6d81} executeQuery: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60308) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_6nj3uy8j_2.t2 as system.query_log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:34.408360 [ 3682 ] {245795f0-2929-4f16-84a8-4f69cdfc6d81} TCPHandler: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:34.485733 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:34.538655 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:34.576069 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:35.506753 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:35.951121 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:35.985012 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:36.268770 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:36.348441 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:36.375379 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:36.956153 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:37.290998 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:37.324448 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:38.005990 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:38.171429 [ 3682 ] {9a0e07ce-52b6-4ab2-9c76-39e30eed4cba} DatabaseReplicated (rdb_test_6nj3uy8j): Database rdb_test_6nj3uy8j exists locally, but its replica does not exist in ZooKeeper (/test/test_6nj3uy8j/rdb/replicas/s1|r1). Assuming it was dropped, will not try to start it up 2025.09.25 21:17:38.218996 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:38.252986 [ 3682 ] {9a0e07ce-52b6-4ab2-9c76-39e30eed4cba} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37558) (comment: 02447_drop_database_replica.sh) (in query: attach database rdb_test_6nj3uy8j), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseOnDisk::iterateMetadataFiles(std::shared_ptr, std::function const&) const @ 0x0000000027cdb3e5 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000027d13ea8 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::TablesLoader::loadTablesAsync(std::unordered_set, std::hash>, std::equal_to>, std::allocator>>) @ 0x00000000291db317 9. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x000000002918eea7 10. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c33d8 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 14. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 18. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:38.306416 [ 3682 ] {9a0e07ce-52b6-4ab2-9c76-39e30eed4cba} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseOnDisk::iterateMetadataFiles(std::shared_ptr, std::function const&) const @ 0x0000000027cdb3e5 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000027d13ea8 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::TablesLoader::loadTablesAsync(std::unordered_set, std::hash>, std::equal_to>, std::allocator>>) @ 0x00000000291db317 9. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x000000002918eea7 10. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c33d8 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 14. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 18. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:38.319644 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:38.760774 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:38.804357 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:38.819969 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:38.856341 [ 3353 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {8b990b7c-a181-415f-9d90-25ec9d0f3599::202509_1_111_22}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:39.320216 [ 3682 ] {42bc2b7b-ba8d-4ba7-a115-bd33da85fb6a} executeQuery: Code: 81. DB::Exception: Database rdb_test_6nj3uy8j does not exist. (UNKNOWN_DATABASE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37564) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_6nj3uy8j.t2 as system.query_log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1471: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ae937 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 7. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 9. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 13. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 14. asan_thread_start(void*) @ 0x000000000aa43059 15. ? @ 0x00007f12cd75eac3 16. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:39.337648 [ 3682 ] {42bc2b7b-ba8d-4ba7-a115-bd33da85fb6a} TCPHandler: Code: 81. DB::Exception: Database rdb_test_6nj3uy8j does not exist. (UNKNOWN_DATABASE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1471: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ae937 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 7. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 9. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 13. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 14. asan_thread_start(void*) @ 0x000000000aa43059 15. ? @ 0x00007f12cd75eac3 16. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:39.350194 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:39.436140 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:39.963070 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:40.190375 [ 3682 ] {4b5d8ec3-8686-4d59-b5be-5eb2bc6dc995} executeQuery: Code: 81. DB::Exception: Database rdb_test_6nj3uy8j does not exist. Maybe you meant rdb_test_6nj3uy8j_2?. (UNKNOWN_DATABASE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37570) (comment: 02447_drop_database_replica.sh) (in query: show tables from rdb_test_6nj3uy8j), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:523: DB::DatabaseCatalog::assertDatabaseExists(String const&) const @ 0x00000000286b3418 5. ./build_docker/./src/Interpreters/InterpreterShowTablesQuery.cpp:0: DB::InterpreterShowTablesQuery::getRewrittenQuery() @ 0x0000000029bcd134 6. ./build_docker/./src/Interpreters/InterpreterShowTablesQuery.cpp:227: DB::InterpreterShowTablesQuery::execute() @ 0x0000000029bce366 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:40.197207 [ 3682 ] {4b5d8ec3-8686-4d59-b5be-5eb2bc6dc995} TCPHandler: Code: 81. DB::Exception: Database rdb_test_6nj3uy8j does not exist. Maybe you meant rdb_test_6nj3uy8j_2?. (UNKNOWN_DATABASE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:523: DB::DatabaseCatalog::assertDatabaseExists(String const&) const @ 0x00000000286b3418 5. ./build_docker/./src/Interpreters/InterpreterShowTablesQuery.cpp:0: DB::InterpreterShowTablesQuery::getRewrittenQuery() @ 0x0000000029bcd134 6. ./build_docker/./src/Interpreters/InterpreterShowTablesQuery.cpp:227: DB::InterpreterShowTablesQuery::execute() @ 0x0000000029bce366 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:40.375086 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:40.416783 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:40.512061 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:41.330700 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:41.343820 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:41.516957 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:41.659088 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:42.075186 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:42.129798 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:42.530191 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:42.611131 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:42.668642 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:42.873616 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:43.449934 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:43.484412 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:43.605904 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:44.069536 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:44.378928 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:44.469984 [ 3682 ] {0a089f73-4f20-43f9-9b9a-e26969540c9d} executeQuery: Code: 81. DB::Exception: Database rdb_test_6nj3uy8j does not exist. Maybe you meant rdb_test_6nj3uy8j_2?. (UNKNOWN_DATABASE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37632) (comment: 02447_drop_database_replica.sh) (in query: drop database rdb_test_6nj3uy8j), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:664: DB::DatabaseCatalog::getDatabase(String const&) const @ 0x00000000286b7273 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f40bc 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:44.472783 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:44.482912 [ 3682 ] {0a089f73-4f20-43f9-9b9a-e26969540c9d} TCPHandler: Code: 81. DB::Exception: Database rdb_test_6nj3uy8j does not exist. Maybe you meant rdb_test_6nj3uy8j_2?. (UNKNOWN_DATABASE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:664: DB::DatabaseCatalog::getDatabase(String const&) const @ 0x00000000286b7273 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f40bc 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:44.511265 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:44.954627 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:45.203981 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:45.244345 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:45.566101 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:45.704194 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:45.807197 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:46.272833 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:46.362711 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:46.500404 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:46.928639 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:46.971196 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:47.331576 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:47.755672 [ 8396 ] {a8db8b5f-1101-46cc-b256-2e88a1235b6c} InterpreterSystemQuery: SYNC REPLICA test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): Timed out. 2025.09.25 21:17:47.778801 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:47.810178 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:47.902729 [ 3364 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {f81dbc1c-a1bd-4968-9514-f0c97137169a::202509_1_79_16}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:47.960039 [ 8396 ] {a8db8b5f-1101-46cc-b256-2e88a1235b6c} executeQuery: Code: 159. DB::Exception: SYNC REPLICA test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): command timed out. See the 'receive_timeout' setting. (TIMEOUT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46454) (comment: 01165_lost_part_empty_partition.sql) (in query: system sync replica rmt2;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1137: DB::InterpreterSystemQuery::syncReplica(DB::ASTSystemQuery&) @ 0x0000000029baeb8e 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:684: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9adf7 6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 7. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 9. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 13. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 14. asan_thread_start(void*) @ 0x000000000aa43059 15. ? @ 0x00007f12cd75eac3 16. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:47.964079 [ 8396 ] {a8db8b5f-1101-46cc-b256-2e88a1235b6c} TCPHandler: Code: 159. DB::Exception: SYNC REPLICA test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): command timed out. See the 'receive_timeout' setting. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1137: DB::InterpreterSystemQuery::syncReplica(DB::ASTSystemQuery&) @ 0x0000000029baeb8e 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:684: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9adf7 6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 7. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 9. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 13. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 14. asan_thread_start(void*) @ 0x000000000aa43059 15. ? @ 0x00007f12cd75eac3 16. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:48.087954 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:48.428523 [ 9743 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:48.836411 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:48.864641 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:48.931101 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:49.462777 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:49.629882 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:49.909847 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:49.939308 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:49.961427 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:50.042168 [ 8396 ] {57b416ba-e2d4-48ac-83c0-012bcfe25e98} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=730). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48056) (comment: 02205_ephemeral_1.sql) (in query: SELECT * FROM t_ephemeral_02205_1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:50.052462 [ 8396 ] {57b416ba-e2d4-48ac-83c0-012bcfe25e98} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=730). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:50.581529 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:50.657150 [ 8396 ] {b1509c22-86b8-451e-8966-f229e42062ff} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=730). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48078) (comment: 00159_parallel_formatting_tsv_and_friends.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c LIMIT 3000000 Format TSVWithNames), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:50.675260 [ 8396 ] {b1509c22-86b8-451e-8966-f229e42062ff} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=730). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:50.741717 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:50.799034 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:51.350662 [ 8396 ] {a8dc77a4-0594-4880-88a1-d078026127c0} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48088) (comment: 02447_drop_database_replica.sh) (in query: drop database rdb_test_6nj3uy8j_4), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:51.358989 [ 8396 ] {a8dc77a4-0594-4880-88a1-d078026127c0} TCPHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:51.370747 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:51.558053 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:51.842199 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:51.881725 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:52.034807 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:52.089352 [ 3682 ] {0b69fccd-bd3e-4838-9a50-2d83404136ed} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=3): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48086) (comment: 02900_window_function_with_sparse_column.sql) (in query: SELECT last_value(value) OVER (PARTITION BY id ORDER BY time ASC) as last_value FROM test1 WHERE (key = 3);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:17:52.108189 [ 3682 ] {0b69fccd-bd3e-4838-9a50-2d83404136ed} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=3): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:17:52.440424 [ 3364 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {f2eeeeb5-70de-4a33-835e-3bff489e4be8::202509_1_69_18}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001adfc17a 4. ./build_docker/./src/Common/MemoryTracker.cpp:187: MemoryTracker::injectFault() const @ 0x000000001adf8d1a 5. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x000000002cc30a65 6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f47b 7. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:52.522286 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:52.590906 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:52.766221 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:52.925633 [ 2871 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:53.010756 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:53.045338 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:53.703252 [ 10192 ] {cdbed463-51db-48b7-82e6-7353fbd086be} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=728). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48130) (comment: 00035_aggregate_keys128.sql) (in query: SELECT SearchEngineID AS k1, AdvEngineID AS k2, count() AS c FROM test.hits GROUP BY k1, k2 ORDER BY c DESC, k1, k2 LIMIT 10 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:53.719748 [ 10192 ] {cdbed463-51db-48b7-82e6-7353fbd086be} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=728). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:53.785201 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:53.921621 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:53.930460 [ 2899 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:53.963849 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:54.166007 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:54.804294 [ 3682 ] {97fd9632-9b0f-4608-aa7f-24f6cd9e8363} executeQuery: Code: 62. DB::Exception: Cannot TRUNCATE dictionary. (SYNTAX_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48106) (comment: 01948_dictionary_quoted_database_name.sql) (in query: DROP DICTIONARY `01945.db`.test_dictionary;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:262: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f1adc 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:54.836010 [ 3682 ] {97fd9632-9b0f-4608-aa7f-24f6cd9e8363} TCPHandler: Code: 62. DB::Exception: Cannot TRUNCATE dictionary. (SYNTAX_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:262: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f1adc 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:54.942892 [ 2924 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:54.943624 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:55.182773 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:55.233167 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:55.476062 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:55.670702 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:55.689397 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:55.734725 [ 10192 ] {18372c3a-d803-492b-86f9-b64ea82a5f15} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48146) (comment: 00061_storage_buffer.sql) (in query: SELECT count() FROM test.hits_buffer;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:55.739597 [ 10192 ] {18372c3a-d803-492b-86f9-b64ea82a5f15} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:55.799938 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:55.946493 [ 2965 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:56.734944 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:56.766154 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:56.779112 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:56.949684 [ 2998 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:57.426550 [ 9743 ] {f314873d-fd2c-4fb1-99e8-07941e556047} executeQuery: Code: 36. DB::Exception: Cannot sleep infinite, very large or negative amount of time (not implemented). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53104) (comment: 01655_sleep_infinite_float.sql) (in query: SELECT sleep(nan);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. DB::FunctionSleep<(DB::FunctionSleepVariant)0>::execute(std::vector> const&, std::shared_ptr const&, bool) const @ 0x000000001820287a 5. DB::FunctionSleep<(DB::FunctionSleepVariant)0>::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000018201734 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 9. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 10. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::ActionsDAG::evaluatePartialResult(std::unordered_map, std::equal_to, std::allocator>>&, std::vector> const&, unsigned long, bool) @ 0x0000000028259428 11. ./contrib/llvm-project/libcxx/include/vector:961: DB::ActionsDAG::updateHeader(DB::Block const&) const @ 0x0000000028256e17 12. ./build_docker/./src/Processors/Transforms/ExpressionTransform.cpp:10: DB::ExpressionTransform::transformHeader(DB::Block const&, DB::ActionsDAG const&) @ 0x000000002d9c2197 13. ./build_docker/./src/Processors/QueryPlan/ExpressionStep.cpp:0: DB::ExpressionStep::ExpressionStep(DB::DataStream const&, DB::ActionsDAG) @ 0x000000002dd8d279 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::(anonymous namespace)::addExpressionStep(DB::QueryPlan&, std::shared_ptr&, String const&, std::unordered_set, std::hash>, std::equal_to>, std::allocator>>&) @ 0x000000002943bd73 15. ./contrib/llvm-project/libcxx/include/string:1499: DB::Planner::buildPlanForQueryNode() @ 0x0000000029434460 16. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 17. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 18. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 19. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 21. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 22. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 24. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 25. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:57.434884 [ 9743 ] {f314873d-fd2c-4fb1-99e8-07941e556047} TCPHandler: Code: 36. DB::Exception: Cannot sleep infinite, very large or negative amount of time (not implemented). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. DB::FunctionSleep<(DB::FunctionSleepVariant)0>::execute(std::vector> const&, std::shared_ptr const&, bool) const @ 0x000000001820287a 5. DB::FunctionSleep<(DB::FunctionSleepVariant)0>::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000018201734 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 9. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 10. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::ActionsDAG::evaluatePartialResult(std::unordered_map, std::equal_to, std::allocator>>&, std::vector> const&, unsigned long, bool) @ 0x0000000028259428 11. ./contrib/llvm-project/libcxx/include/vector:961: DB::ActionsDAG::updateHeader(DB::Block const&) const @ 0x0000000028256e17 12. ./build_docker/./src/Processors/Transforms/ExpressionTransform.cpp:10: DB::ExpressionTransform::transformHeader(DB::Block const&, DB::ActionsDAG const&) @ 0x000000002d9c2197 13. ./build_docker/./src/Processors/QueryPlan/ExpressionStep.cpp:0: DB::ExpressionStep::ExpressionStep(DB::DataStream const&, DB::ActionsDAG) @ 0x000000002dd8d279 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::(anonymous namespace)::addExpressionStep(DB::QueryPlan&, std::shared_ptr&, String const&, std::unordered_set, std::hash>, std::equal_to>, std::allocator>>&) @ 0x000000002943bd73 15. ./contrib/llvm-project/libcxx/include/string:1499: DB::Planner::buildPlanForQueryNode() @ 0x0000000029434460 16. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 17. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 18. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 19. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 21. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 22. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 24. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 25. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:57.532684 [ 9743 ] {44a23552-0e6f-4b2d-afa6-35cac0dedb53} executeQuery: Code: 36. DB::Exception: Cannot sleep infinite, very large or negative amount of time (not implemented). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53104) (comment: 01655_sleep_infinite_float.sql) (in query: SELECT sleep(inf);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. DB::FunctionSleep<(DB::FunctionSleepVariant)0>::execute(std::vector> const&, std::shared_ptr const&, bool) const @ 0x000000001820287a 5. DB::FunctionSleep<(DB::FunctionSleepVariant)0>::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000018201734 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 9. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 10. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::ActionsDAG::evaluatePartialResult(std::unordered_map, std::equal_to, std::allocator>>&, std::vector> const&, unsigned long, bool) @ 0x0000000028259428 11. ./contrib/llvm-project/libcxx/include/vector:961: DB::ActionsDAG::updateHeader(DB::Block const&) const @ 0x0000000028256e17 12. ./build_docker/./src/Processors/Transforms/ExpressionTransform.cpp:10: DB::ExpressionTransform::transformHeader(DB::Block const&, DB::ActionsDAG const&) @ 0x000000002d9c2197 13. ./build_docker/./src/Processors/QueryPlan/ExpressionStep.cpp:0: DB::ExpressionStep::ExpressionStep(DB::DataStream const&, DB::ActionsDAG) @ 0x000000002dd8d279 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::(anonymous namespace)::addExpressionStep(DB::QueryPlan&, std::shared_ptr&, String const&, std::unordered_set, std::hash>, std::equal_to>, std::allocator>>&) @ 0x000000002943bd73 15. ./contrib/llvm-project/libcxx/include/string:1499: DB::Planner::buildPlanForQueryNode() @ 0x0000000029434460 16. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 17. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 18. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 19. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 21. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 22. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 24. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 25. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:57.546651 [ 9743 ] {44a23552-0e6f-4b2d-afa6-35cac0dedb53} TCPHandler: Code: 36. DB::Exception: Cannot sleep infinite, very large or negative amount of time (not implemented). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. DB::FunctionSleep<(DB::FunctionSleepVariant)0>::execute(std::vector> const&, std::shared_ptr const&, bool) const @ 0x000000001820287a 5. DB::FunctionSleep<(DB::FunctionSleepVariant)0>::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000018201734 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 9. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 10. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::ActionsDAG::evaluatePartialResult(std::unordered_map, std::equal_to, std::allocator>>&, std::vector> const&, unsigned long, bool) @ 0x0000000028259428 11. ./contrib/llvm-project/libcxx/include/vector:961: DB::ActionsDAG::updateHeader(DB::Block const&) const @ 0x0000000028256e17 12. ./build_docker/./src/Processors/Transforms/ExpressionTransform.cpp:10: DB::ExpressionTransform::transformHeader(DB::Block const&, DB::ActionsDAG const&) @ 0x000000002d9c2197 13. ./build_docker/./src/Processors/QueryPlan/ExpressionStep.cpp:0: DB::ExpressionStep::ExpressionStep(DB::DataStream const&, DB::ActionsDAG) @ 0x000000002dd8d279 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::(anonymous namespace)::addExpressionStep(DB::QueryPlan&, std::shared_ptr&, String const&, std::unordered_set, std::hash>, std::equal_to>, std::allocator>>&) @ 0x000000002943bd73 15. ./contrib/llvm-project/libcxx/include/string:1499: DB::Planner::buildPlanForQueryNode() @ 0x0000000029434460 16. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 17. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 18. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 19. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 20. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 21. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 22. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 23. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 24. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 25. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:17:57.706876 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:57.775308 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:57.833922 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:57.952777 [ 3027 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:58.382664 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:58.897426 [ 9743 ] {6cb107df-1df7-4781-a922-8200b5fdb49e} executeQuery: Code: 43. DB::Exception: Argument 'max_width' must not be NaN: In scope SELECT bar((greatCircleAngle(65537, 2, 1, 1) - 1) * 65535, 1048576, 1048577, nan). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53142) (comment: 01621_bar_nan_arguments.sql) (in query: SELECT bar((greatCircleAngle(65537, 2, 1, 1) - 1) * 65535, 1048576, 1048577, nan);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. DB::(anonymous namespace)::FunctionBar::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000d0e9429 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:17:58.900630 [ 9743 ] {6cb107df-1df7-4781-a922-8200b5fdb49e} TCPHandler: Code: 43. DB::Exception: Argument 'max_width' must not be NaN: In scope SELECT bar((greatCircleAngle(65537, 2, 1, 1) - 1) * 65535, 1048576, 1048577, nan). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. DB::(anonymous namespace)::FunctionBar::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000d0e9429 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:17:58.956354 [ 3040 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:59.006024 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:59.012369 [ 9743 ] {0f734710-e2cc-46a3-b200-10f5854645b7} executeQuery: Code: 43. DB::Exception: Argument 'max_width' must not be NaN: In scope SELECT bar(1, 1, 1, nan). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53142) (comment: 01621_bar_nan_arguments.sql) (in query: select bar(1,1,1,nan);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. DB::(anonymous namespace)::FunctionBar::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000d0e9429 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:17:59.018742 [ 9743 ] {0f734710-e2cc-46a3-b200-10f5854645b7} TCPHandler: Code: 43. DB::Exception: Argument 'max_width' must not be NaN: In scope SELECT bar(1, 1, 1, nan). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. DB::(anonymous namespace)::FunctionBar::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000d0e9429 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:17:59.061057 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:59.366676 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:59.496412 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:17:59.622570 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:00.055563 [ 3068 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:00.216722 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:00.735870 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:00.760654 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:00.899677 [ 10192 ] {2eae5b64-5a7a-466a-907b-5e97decbb4cc} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53230) (comment: 00082_quantiles.sql) (in query: SELECT CounterID AS k, quantilesExact(0.1, 0.5, 0.9, 0.99, 0.999)(ResolutionWidth) FROM test.hits GROUP BY k ORDER BY count() DESC, CounterID LIMIT 10;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:00.924225 [ 10192 ] {2eae5b64-5a7a-466a-907b-5e97decbb4cc} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:00.953419 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:01.059873 [ 3094 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:01.283988 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:01.323097 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:01.390888 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:02.063479 [ 3122 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:02.079424 [ 10192 ] {faabd827-5e76-4358-bd00-8fedb375234c} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_knhvna7w, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53286) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from database rdb_test_knhvna7w), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./src/Databases/DatabaseReplicated.h:76: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab2fb 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:02.096173 [ 10192 ] {faabd827-5e76-4358-bd00-8fedb375234c} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_knhvna7w, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./src/Databases/DatabaseReplicated.h:76: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab2fb 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:02.127974 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:02.418872 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:02.453571 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:02.712908 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:02.869759 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:02.934419 [ 10544 ] {92a8b367-ade9-4822-87c4-f2cd96049b6b} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_knhvna7w, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53296) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 'r1' from shard 's1' from database rdb_test_knhvna7w), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./src/Databases/DatabaseReplicated.h:76: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab2fb 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:02.953512 [ 10544 ] {92a8b367-ade9-4822-87c4-f2cd96049b6b} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_knhvna7w, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./src/Databases/DatabaseReplicated.h:76: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab2fb 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:03.066782 [ 3155 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:03.312411 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:03.678386 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:03.740742 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:03.978303 [ 3682 ] {a0bded91-384b-42d4-b19d-873a0f8fafe2} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_knhvna7w, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53320) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from zkpath '/test/test_knhvna7w/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:03.982890 [ 3682 ] {a0bded91-384b-42d4-b19d-873a0f8fafe2} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_knhvna7w, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:04.012707 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:04.069438 [ 3183 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:04.079575 [ 10192 ] {5431a319-da43-49c8-bcf9-a5b77b33d95c} executeQuery: Code: 57. DB::Exception: Table test_9.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53332) (comment: 00161_parallel_parsing_with_names.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:04.085846 [ 10192 ] {5431a319-da43-49c8-bcf9-a5b77b33d95c} TCPHandler: Code: 57. DB::Exception: Table test_9.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:04.413631 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:04.446289 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:04.519888 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:04.636154 [ 10536 ] {567a5597-809e-40a2-a11f-e2ef79f1fa80} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=734). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:46568) (comment: 00149_quantiles_timing_distributed.sql) (in query: SELECT `__table1`.`CounterID` AS `CounterID`, quantileTiming(0.5)(`__table1`.`SendTiming`) AS `quantileTiming(0.5)(SendTiming)`, count() AS `count()` FROM `test`.`hits` AS `__table1` WHERE `__table1`.`SendTiming` != -1 GROUP BY `__table1`.`CounterID`), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:04.651028 [ 10536 ] {567a5597-809e-40a2-a11f-e2ef79f1fa80} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=734). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:04.913816 [ 10342 ] {669f5f26-0273-42d8-a24b-779927d6affa} executeQuery: Code: 439. DB::Exception: Received from 127.0.0.6:9000. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=734). Stack trace: 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 : While executing Remote. (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53274) (comment: 00149_quantiles_timing_distributed.sql) (in query: SELECT sum(cityHash64(*)) FROM (SELECT CounterID, quantileTiming(0.5)(SendTiming), count() FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) WHERE SendTiming != -1 GROUP BY CounterID) SETTINGS optimize_aggregation_in_order = 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001aeb974e 3. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000002cee6c6a 4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x000000002cef780e 5. ./build_docker/./src/Client/PacketReceiver.cpp:0: DB::PacketReceiver::Task::run(std::function, std::function) @ 0x000000002cf9767e 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278ab436 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278aaa44 8. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func>>(unsigned int, unsigned int) @ 0x00000000278a8394 2025.09.25 21:18:04.936465 [ 10543 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::ffff:127.0.0.1]:57348, local: [::ffff:127.0.0.9]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:04.947665 [ 10539 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::ffff:127.0.0.1]:55132, local: [::ffff:127.0.0.5]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:04.953480 [ 10192 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::ffff:127.0.0.1]:35330, local: [::ffff:127.0.0.4]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:04.962657 [ 10537 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::ffff:127.0.0.1]:52232, local: [::ffff:127.0.0.3]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:04.972354 [ 3682 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::ffff:127.0.0.1]:49722, local: [::ffff:127.0.0.2]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:04.975838 [ 10342 ] {669f5f26-0273-42d8-a24b-779927d6affa} TCPHandler: Code: 439. DB::Exception: Received from 127.0.0.6:9000. DB::Exception: Cannot schedule a task: fault injected (threads=759, jobs=734). Stack trace: 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 : While executing Remote. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001aeb974e 3. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000002cee6c6a 4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x000000002cef780e 5. ./build_docker/./src/Client/PacketReceiver.cpp:0: DB::PacketReceiver::Task::run(std::function, std::function) @ 0x000000002cf9767e 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278ab436 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278aaa44 8. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func>>(unsigned int, unsigned int) @ 0x00000000278a8394 2025.09.25 21:18:05.025531 [ 10536 ] {b12b2f5e-f147-4f4e-98e5-59d0301d769d} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_knhvna7w, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53342) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from zkpath '/test/test_knhvna7w/rdb/'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:05.053860 [ 10536 ] {b12b2f5e-f147-4f4e-98e5-59d0301d769d} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_knhvna7w, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:05.072361 [ 3190 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:05.156486 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:05.206434 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:05.439524 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:05.996299 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:06.060302 [ 3682 ] {c1abf689-639f-4acc-bc02-e7e2db73b4ae} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_knhvna7w, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53362) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 'r1' from shard 's1' from zkpath '/test/test_knhvna7w/rdb/'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:06.074487 [ 3243 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:06.080846 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:06.081082 [ 3682 ] {c1abf689-639f-4acc-bc02-e7e2db73b4ae} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_knhvna7w, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:06.118540 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:06.672084 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:06.728106 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:06.875547 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:07.026232 [ 10342 ] {db906e0e-ed8b-494f-a012-f12d5fb679c5} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=0): (while reading column StartURL): (while reading from part /var/lib/clickhouse/store/fab/fab61230-e14a-49b2-a373-9edad6fb4784/201403_1_10_1/ in table test.visits (fab61230-e14a-49b2-a373-9edad6fb4784) located on disk default of type local, from mark 125 with max_rows_to_read = 13984): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53356) (comment: 00044_any_left_join_string.sql) (in query: SELECT domain, hits, visits FROM ( SELECT domain(URL) AS domain, count() AS hits FROM test.hits GROUP BY domain ) ANY LEFT JOIN ( SELECT domain(StartURL) AS domain, sum(Sign) AS visits FROM test.visits GROUP BY domain ) USING domain ORDER BY hits DESC LIMIT 10 SETTINGS joined_subquery_requires_alias = 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x000000000b581216 13. ./build_docker/./src/Compression/CompressedReadBufferBase.cpp:0: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x000000002775f390 14. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:22: DB::CompressedReadBufferFromFile::nextImpl() @ 0x0000000027763d41 15. DB::ReadBuffer::next() @ 0x000000000b528aeb 16. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:159: void DB::deserializeBinarySSE2<4>(DB::PODArray, 63ul, 64ul>&, DB::PODArray, 63ul, 64ul>&, DB::ReadBuffer&, unsigned long) @ 0x0000000027c0f641 17. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:0: DB::SerializationString::deserializeBinaryBulk(DB::IColumn&, DB::ReadBuffer&, unsigned long, double) const @ 0x0000000027c0ecfa 18. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027aa00db 19. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:196: DB::MergeTreeReaderWide::readData(DB::NameAndTypePair const&, std::shared_ptr const&, COW::immutable_ptr&, unsigned long, bool, unsigned long, unsigned long, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>&, bool) @ 0x000000002c80211e 20. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:158: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c800e8d 21. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 22. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000002c7e0473 23. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbec3 24. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 25. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:38: DB::MergeTreeThreadSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002de66094 26. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 27. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 28. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 29. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 30. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 31. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 2025.09.25 21:18:07.062944 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:07.077175 [ 2766 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:07.100716 [ 3682 ] {7aff08f5-b021-40b0-912c-42ce3818fe54} executeQuery: Code: 36. DB::Exception: Path /test/test_knhvna7w does not look like a path of Replicated database. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42822) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from zkpath '/test/test_knhvna7w/'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1501: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cbfc 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:07.119439 [ 3682 ] {7aff08f5-b021-40b0-912c-42ce3818fe54} TCPHandler: Code: 36. DB::Exception: Path /test/test_knhvna7w does not look like a path of Replicated database. (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1501: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cbfc 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:07.123555 [ 10342 ] {db906e0e-ed8b-494f-a012-f12d5fb679c5} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=0): (while reading column StartURL): (while reading from part /var/lib/clickhouse/store/fab/fab61230-e14a-49b2-a373-9edad6fb4784/201403_1_10_1/ in table test.visits (fab61230-e14a-49b2-a373-9edad6fb4784) located on disk default of type local, from mark 125 with max_rows_to_read = 13984): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x000000000b581216 13. ./build_docker/./src/Compression/CompressedReadBufferBase.cpp:0: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x000000002775f390 14. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:22: DB::CompressedReadBufferFromFile::nextImpl() @ 0x0000000027763d41 15. DB::ReadBuffer::next() @ 0x000000000b528aeb 16. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:159: void DB::deserializeBinarySSE2<4>(DB::PODArray, 63ul, 64ul>&, DB::PODArray, 63ul, 64ul>&, DB::ReadBuffer&, unsigned long) @ 0x0000000027c0f641 17. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:0: DB::SerializationString::deserializeBinaryBulk(DB::IColumn&, DB::ReadBuffer&, unsigned long, double) const @ 0x0000000027c0ecfa 18. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027aa00db 19. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:196: DB::MergeTreeReaderWide::readData(DB::NameAndTypePair const&, std::shared_ptr const&, COW::immutable_ptr&, unsigned long, bool, unsigned long, unsigned long, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>&, bool) @ 0x000000002c80211e 20. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:158: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c800e8d 21. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 22. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000002c7e0473 23. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbec3 24. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 25. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:38: DB::MergeTreeThreadSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002de66094 26. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 27. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 28. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 29. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 30. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 31. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 2025.09.25 21:18:07.628404 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:07.721588 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:08.080547 [ 2777 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:08.137082 [ 3682 ] {c8a4c2d5-11ed-410d-a614-304a709c6dde} executeQuery: Code: 36. DB::Exception: Replica s2|r1 does not exist (database path: /test/test_knhvna7w/rdb). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42836) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's2|r1' from zkpath '/test/test_knhvna7w/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:08.150931 [ 3682 ] {c8a4c2d5-11ed-410d-a614-304a709c6dde} TCPHandler: Code: 36. DB::Exception: Replica s2|r1 does not exist (database path: /test/test_knhvna7w/rdb). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:08.210286 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:08.324263 [ 10536 ] {8dd43f63-7e32-49c4-833d-0269a6c34138} executeQuery: Code: 57. DB::Exception: Table test_9.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42858) (comment: 00161_parallel_parsing_with_names.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:08.358214 [ 10536 ] {8dd43f63-7e32-49c4-833d-0269a6c34138} TCPHandler: Code: 57. DB::Exception: Table test_9.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:08.844780 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:08.900027 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:09.048681 [ 3682 ] {caf59a6f-4589-4252-9811-745e3e02b4f7} executeQuery: Code: 36. DB::Exception: Replica r1|s1 does not exist (database path: /test/test_knhvna7w/rdb). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42862) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1' from shard 'r1' from zkpath '/test/test_knhvna7w/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:09.084269 [ 2799 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:09.090898 [ 3682 ] {caf59a6f-4589-4252-9811-745e3e02b4f7} TCPHandler: Code: 36. DB::Exception: Replica r1|s1 does not exist (database path: /test/test_knhvna7w/rdb). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:09.167072 [ 9743 ] {0acbf697-43d0-47ca-b7f3-c27bd716e777} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42840) (comment: 00095_hyperscan_profiler.sql) (in query: SELECT count() FROM test.hits WHERE multiFuzzyMatchAny(URL, 2, ['about/address', 'for_woman', '^https?://lm-company.ruy/$', 'ultimateguitar.com']);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:09.172608 [ 9743 ] {0acbf697-43d0-47ca-b7f3-c27bd716e777} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:09.233968 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:09.375220 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:09.389727 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:09.416794 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:09.832637 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:09.978963 [ 3682 ] {b97f3b55-e0f8-4e57-b03e-60270687c5af} executeQuery: Code: 36. DB::Exception: Replica s1|s1|r1 does not exist (database path: /test/test_knhvna7w/rdb). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42890) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from shard 's1' from zkpath '/test/test_knhvna7w/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:09.994425 [ 3682 ] {b97f3b55-e0f8-4e57-b03e-60270687c5af} TCPHandler: Code: 36. DB::Exception: Replica s1|s1|r1 does not exist (database path: /test/test_knhvna7w/rdb). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:10.087337 [ 2823 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:10.092497 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:10.625727 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:10.774958 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:10.905015 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:11.068288 [ 3682 ] {d5216a46-3dca-4c7f-9794-56e91810a89a} executeQuery: Code: 36. DB::Exception: Invalid replica name, '/' is not allowed: s2/r1. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42922) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's2/r1' from zkpath '/test/test_knhvna7w/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x000000000aafc574 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1492: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cca3 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:11.087835 [ 3682 ] {d5216a46-3dca-4c7f-9794-56e91810a89a} TCPHandler: Code: 36. DB::Exception: Invalid replica name, '/' is not allowed: s2/r1. (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x000000000aafc574 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1492: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cca3 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:11.090560 [ 2857 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:11.282630 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:11.722373 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:11.741679 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:11.758985 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:11.908152 [ 9743 ] {2bde662d-8c0d-4520-885e-b6d8382ef135} executeQuery: Code: 57. DB::Exception: Table test_9.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42952) (comment: 00161_parallel_parsing_with_names.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:11.913004 [ 9743 ] {2bde662d-8c0d-4520-885e-b6d8382ef135} TCPHandler: Code: 57. DB::Exception: Table test_9.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:12.092697 [ 2885 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:12.092857 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:12.119018 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:12.347524 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:12.362636 [ 5075 ] {} DDLWorker(rdb_test_knhvna7w_2): Error on initialization of rdb_test_knhvna7w_2: Code: 57. DB::Exception: Directory for table data store/ed6/ed60a55a-bec7-4319-8001-d68d9580a181/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:12.715790 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:12.800281 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:13.095864 [ 2912 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:13.227083 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:13.257177 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:13.492868 [ 8174 ] {} DDLWorker(rdb_test_knhvna7w_3): Error on initialization of rdb_test_knhvna7w_3: Code: 57. DB::Exception: Directory for table data store/ed6/ed60a55a-bec7-4319-8001-d68d9580a181/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:13.512485 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:13.729287 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:13.819940 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:14.099212 [ 2940 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:14.186346 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:14.621753 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:14.653356 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:14.654665 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:15.103238 [ 2964 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:15.346722 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:15.386304 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:15.386497 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:16.106145 [ 2990 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:16.264791 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:16.443407 [ 3682 ] {54b4cef9-1ee3-40a1-911f-8a47acbe7f57} executeQuery: Code: 36. DB::Exception: Replica s1|r1 is active, cannot drop it (database path: /test/test_knhvna7w/rdb). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43092) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from database rdb_test_knhvna7w_2), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1513: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cde0 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab31e 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:16.458784 [ 3682 ] {54b4cef9-1ee3-40a1-911f-8a47acbe7f57} TCPHandler: Code: 36. DB::Exception: Replica s1|r1 is active, cannot drop it (database path: /test/test_knhvna7w/rdb). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1513: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cde0 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab31e 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:16.702304 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:16.767797 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:16.839762 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:17.109093 [ 3022 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:17.526717 [ 5075 ] {} DDLWorker(rdb_test_knhvna7w_2): Error on initialization of rdb_test_knhvna7w_2: Code: 57. DB::Exception: Directory for table data store/ed6/ed60a55a-bec7-4319-8001-d68d9580a181/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:17.803212 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:17.853752 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:17.876468 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:18.113517 [ 3051 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:18.504915 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:18.513263 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:18.571495 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:18.636024 [ 8174 ] {} DDLWorker(rdb_test_knhvna7w_3): Error on initialization of rdb_test_knhvna7w_3: Code: 57. DB::Exception: Directory for table data store/824/8248bbc9-88a8-496a-b3ff-456b4b992042/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:19.125583 [ 3071 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:19.207776 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:19.744585 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:19.874196 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:19.874491 [ 10536 ] {6a9119a6-b812-4ea0-baf2-457403f16253} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=2): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38180) (comment: 00181_cross_join_compression.sql) (in query: SELECT CounterID, StartURL FROM unit, test.visits ORDER BY (CounterID, StartURL) DESC LIMIT 1000;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:144: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002def1bcc 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 22. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 23. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:18:19.899483 [ 10536 ] {6a9119a6-b812-4ea0-baf2-457403f16253} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=13, jobs=2): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:144: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002def1bcc 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 22. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 23. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 24. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:18:19.945418 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:20.128511 [ 3101 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:20.954426 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:21.050785 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:21.108475 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:21.130686 [ 3114 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:21.853195 [ 5082 ] {} DDLWorker(rdb_test_knhvna7w_9): Error on initialization of rdb_test_knhvna7w_9: Code: 57. DB::Exception: Directory for table data store/165/16526ff2-4d90-41a8-bbc7-02863b3e1a98/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:21.934635 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:22.132933 [ 3140 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:22.199327 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:22.327197 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:22.668961 [ 5075 ] {} DDLWorker(rdb_test_knhvna7w_2): Error on initialization of rdb_test_knhvna7w_2: Code: 57. DB::Exception: Directory for table data store/d9b/d9b57fc1-bbe7-4c06-9ca0-ab88ff19caaf/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:22.699364 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:22.746429 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:23.044652 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:23.136226 [ 3170 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:23.554785 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:23.590221 [ 9743 ] {304d90df-4c20-4446-a02b-88db5ef0c329} executeQuery: Code: 452. DB::Exception: Setting allow_experimental_analyzer should not be changed. (SETTING_CONSTRAINT_VIOLATION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38248) (comment: 00172_early_constant_folding.sql) (in query: EXPLAIN PIPELINE SELECT count(JavaEnable) FROM test.hits WHERE WatchID = 1 OR Title = 'next' OR URL = 'prev' OR URL = '???' OR 1 SETTINGS enable_analyzer = 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges const&, DB::SettingSource) const @ 0x0000000026f807e3 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraints(DB::SettingsChanges const&, DB::SettingSource) @ 0x000000002853b6ad 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::executeForCurrentContext(bool) @ 0x00000000294f6701 9. ./src/Interpreters/InterpreterSetQuery.h:14: DB::applySettingsFromSelectWithUnion(DB::ASTSelectWithUnionQuery const&, std::shared_ptr) @ 0x00000000294f7ccd 10. ./build_docker/./src/Interpreters/InterpreterSetQuery.cpp:65: DB::InterpreterSetQuery::applySettingsFromQuery(std::shared_ptr const&, std::shared_ptr) @ 0x00000000294f6e01 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::applySettingsFromQuery(std::shared_ptr const&, std::shared_ptr) @ 0x00000000294f7439 12. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b33ca4 13. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 15. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 19. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 20. asan_thread_start(void*) @ 0x000000000aa43059 21. ? @ 0x00007f12cd75eac3 22. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:23.615789 [ 9743 ] {304d90df-4c20-4446-a02b-88db5ef0c329} TCPHandler: Code: 452. DB::Exception: Setting allow_experimental_analyzer should not be changed. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges const&, DB::SettingSource) const @ 0x0000000026f807e3 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraints(DB::SettingsChanges const&, DB::SettingSource) @ 0x000000002853b6ad 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::executeForCurrentContext(bool) @ 0x00000000294f6701 9. ./src/Interpreters/InterpreterSetQuery.h:14: DB::applySettingsFromSelectWithUnion(DB::ASTSelectWithUnionQuery const&, std::shared_ptr) @ 0x00000000294f7ccd 10. ./build_docker/./src/Interpreters/InterpreterSetQuery.cpp:65: DB::InterpreterSetQuery::applySettingsFromQuery(std::shared_ptr const&, std::shared_ptr) @ 0x00000000294f6e01 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::applySettingsFromQuery(std::shared_ptr const&, std::shared_ptr) @ 0x00000000294f7439 12. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b33ca4 13. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 15. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 19. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 20. asan_thread_start(void*) @ 0x000000000aa43059 21. ? @ 0x00007f12cd75eac3 22. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:23.642357 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:23.780998 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:24.053304 [ 8174 ] {} DDLWorker(rdb_test_knhvna7w_3): Error on initialization of rdb_test_knhvna7w_3: Code: 57. DB::Exception: Directory for table data store/d9b/d9b57fc1-bbe7-4c06-9ca0-ab88ff19caaf/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:24.139041 [ 3200 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:24.683290 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:24.712568 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:24.734247 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:25.141135 [ 3190 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:25.344548 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:25.399872 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:25.403338 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:25.567424 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:26.116004 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:26.144095 [ 3246 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:26.221406 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:26.351384 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:26.353609 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:26.368495 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:26.604914 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:26.761112 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:27.034278 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:27.093698 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:27.153663 [ 2755 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:27.343978 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:27.655291 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:27.689900 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:27.779706 [ 5075 ] {} DDLWorker(rdb_test_knhvna7w_2): Error on initialization of rdb_test_knhvna7w_2: Code: 57. DB::Exception: Directory for table data store/d9b/d9b57fc1-bbe7-4c06-9ca0-ab88ff19caaf/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:28.028082 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:28.156546 [ 2797 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:28.598546 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:28.713756 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:28.751960 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:29.140233 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:29.159747 [ 2807 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:29.213871 [ 8174 ] {} DDLWorker(rdb_test_knhvna7w_3): Error on initialization of rdb_test_knhvna7w_3: Code: 57. DB::Exception: Directory for table data store/165/16526ff2-4d90-41a8-bbc7-02863b3e1a98/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:29.653419 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:29.934814 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:29.951228 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:30.162593 [ 2848 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:30.273264 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:30.946891 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:31.061175 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:31.092417 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:31.165614 [ 2873 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:31.495152 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:32.108440 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:32.126086 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:32.216087 [ 2902 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:32.443068 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:32.745105 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:32.763415 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:32.815173 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:33.004222 [ 5075 ] {} DDLWorker(rdb_test_knhvna7w_2): Error on initialization of rdb_test_knhvna7w_2: Code: 57. DB::Exception: Directory for table data store/d9b/d9b57fc1-bbe7-4c06-9ca0-ab88ff19caaf/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:33.062589 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:33.228759 [ 2925 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:33.508635 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:33.560622 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:33.642331 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:34.082349 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:34.115115 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:34.236058 [ 2958 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:34.372286 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:34.451004 [ 8174 ] {} DDLWorker(rdb_test_knhvna7w_3): Error on initialization of rdb_test_knhvna7w_3: Code: 57. DB::Exception: Directory for table data store/165/16526ff2-4d90-41a8-bbc7-02863b3e1a98/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:34.562035 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:34.952480 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:35.004676 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:35.017975 [ 10931 ] {425ef971-98da-49ee-a94b-f0da3a16aa56} executeQuery: Code: 159. DB::Exception: Distributed DDL task /test/test_knhvna7w/rdb/log/query-0000000009 is not finished on 3 of 4 hosts (0 of them are currently executing the task, 2 are inactive). They are going to execute the query in background. Was waiting for 5.07595719 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42608) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_knhvna7w.t22 (n int) engine=Log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::DDLQueryStatusSource::generate() @ 0x0000000029b11829 4. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 5. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 10. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d3463e7 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:35.021211 [ 10931 ] {425ef971-98da-49ee-a94b-f0da3a16aa56} TCPHandler: Code: 159. DB::Exception: Distributed DDL task /test/test_knhvna7w/rdb/log/query-0000000009 is not finished on 3 of 4 hosts (0 of them are currently executing the task, 2 are inactive). They are going to execute the query in background. Was waiting for 5.07595719 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::DDLQueryStatusSource::generate() @ 0x0000000029b11829 4. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 5. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 10. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d3463e7 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:35.162898 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:35.242858 [ 2982 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:35.541165 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:35.847019 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:35.871846 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:36.250017 [ 3005 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:36.396180 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:36.505469 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:36.505933 [ 10537 ] {c8f66fef-ec92-47cf-8116-ce67ae0c2c0b} executeQuery: Code: 394. DB::Exception: Query was cancelled: while executing 'FUNCTION sleep(1_UInt8 :: 0) -> sleep(1_UInt8) UInt8 : 1'. (QUERY_WAS_CANCELLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42738) (comment: 00909_kill_not_initialized_query.sh) (in query: SELECT sum(1) FROM cannot_kill_query WHERE NOT ignore(sleep(1)) SETTINGS max_threads = 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. DB::FunctionSleep<(DB::FunctionSleepVariant)0>::execute(std::vector> const&, std::shared_ptr const&, bool) const @ 0x0000000018202475 6. DB::FunctionSleep<(DB::FunctionSleepVariant)0>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000018201711 7. DB::FunctionToExecutableFunctionAdaptor::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac9135 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccce5b 9. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 10. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 11. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::ExpressionActions::execute(DB::Block&, unsigned long&, bool, bool) const @ 0x00000000286fc2bd 12. ./contrib/llvm-project/libcxx/include/vector:547: DB::ExpressionActions::execute(DB::Block&, bool, bool) const @ 0x00000000286ff41d 13. ./build_docker/./src/Storages/VirtualColumnUtils.cpp:102: DB::VirtualColumnUtils::filterBlockWithExpression(std::shared_ptr const&, DB::Block&) @ 0x000000002bdca21e 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::VirtualColumnUtils::filterBlockWithPredicate(DB::ActionsDAG::Node const*, DB::Block&, std::shared_ptr, bool) @ 0x000000002bdda766 15. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:6950: DB::MergeTreeData::getMinMaxCountProjectionBlock(std::shared_ptr const&, std::vector> const&, DB::ActionsDAG const*, std::vector, std::allocator>> const&, std::unordered_map, std::equal_to, std::allocator>> const*, std::shared_ptr) const @ 0x000000002c50a88f 16. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizeUseAggregateProjection.cpp:480: DB::QueryPlanOptimizations::getAggregateProjectionCandidates(DB::QueryPlan::Node&, DB::AggregatingStep&, DB::ReadFromMergeTree&, std::shared_ptr, std::equal_to, std::allocator>>> const&, bool) @ 0x000000002dfa3aaa 17. ./src/Processors/QueryPlan/ReadFromMergeTree.h:187: DB::QueryPlanOptimizations::optimizeUseAggregateProjections(DB::QueryPlan::Node&, std::list>&, bool) @ 0x000000002dfa778f 18. ./contrib/llvm-project/libcxx/include/optional:344: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list>&) @ 0x000000002df631d6 19. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:507: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x000000002de001ca 20. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:36: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddfec4b 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 23. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 24. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 26. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 27. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 29. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 30. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:18:36.528459 [ 10537 ] {c8f66fef-ec92-47cf-8116-ce67ae0c2c0b} TCPHandler: Code: 394. DB::Exception: Query was cancelled: while executing 'FUNCTION sleep(1_UInt8 :: 0) -> sleep(1_UInt8) UInt8 : 1'. (QUERY_WAS_CANCELLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/ProcessList.cpp:480: DB::QueryStatus::checkTimeLimit() @ 0x00000000295fbb84 5. DB::FunctionSleep<(DB::FunctionSleepVariant)0>::execute(std::vector> const&, std::shared_ptr const&, bool) const @ 0x0000000018202475 6. DB::FunctionSleep<(DB::FunctionSleepVariant)0>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x0000000018201711 7. DB::FunctionToExecutableFunctionAdaptor::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac9135 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccce5b 9. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 10. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 11. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::ExpressionActions::execute(DB::Block&, unsigned long&, bool, bool) const @ 0x00000000286fc2bd 12. ./contrib/llvm-project/libcxx/include/vector:547: DB::ExpressionActions::execute(DB::Block&, bool, bool) const @ 0x00000000286ff41d 13. ./build_docker/./src/Storages/VirtualColumnUtils.cpp:102: DB::VirtualColumnUtils::filterBlockWithExpression(std::shared_ptr const&, DB::Block&) @ 0x000000002bdca21e 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::VirtualColumnUtils::filterBlockWithPredicate(DB::ActionsDAG::Node const*, DB::Block&, std::shared_ptr, bool) @ 0x000000002bdda766 15. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:6950: DB::MergeTreeData::getMinMaxCountProjectionBlock(std::shared_ptr const&, std::vector> const&, DB::ActionsDAG const*, std::vector, std::allocator>> const&, std::unordered_map, std::equal_to, std::allocator>> const*, std::shared_ptr) const @ 0x000000002c50a88f 16. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizeUseAggregateProjection.cpp:480: DB::QueryPlanOptimizations::getAggregateProjectionCandidates(DB::QueryPlan::Node&, DB::AggregatingStep&, DB::ReadFromMergeTree&, std::shared_ptr, std::equal_to, std::allocator>>> const&, bool) @ 0x000000002dfa3aaa 17. ./src/Processors/QueryPlan/ReadFromMergeTree.h:187: DB::QueryPlanOptimizations::optimizeUseAggregateProjections(DB::QueryPlan::Node&, std::list>&, bool) @ 0x000000002dfa778f 18. ./contrib/llvm-project/libcxx/include/optional:344: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list>&) @ 0x000000002df631d6 19. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:507: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x000000002de001ca 20. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:36: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddfec4b 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 23. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 24. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 26. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 27. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 29. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 30. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:18:36.530338 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:37.212045 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:37.253378 [ 3033 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:37.710709 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:37.823021 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:38.061739 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:38.234078 [ 5075 ] {} DDLWorker(rdb_test_knhvna7w_2): Error on initialization of rdb_test_knhvna7w_2: Code: 57. DB::Exception: Directory for table data store/d9b/d9b57fc1-bbe7-4c06-9ca0-ab88ff19caaf/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:38.256579 [ 3054 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:38.701158 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:38.729039 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:39.046296 [ 10538 ] {9d4a6346-c072-4e42-a24e-b98cf85585b9} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42748) (comment: 00024_random_counters.sql) (in query: SELECT uniq(UserID), sum(Sign) FROM test.visits WHERE CounterID = 9627212;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:39.054424 [ 10538 ] {9d4a6346-c072-4e42-a24e-b98cf85585b9} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:39.103792 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:39.306827 [ 3073 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:39.553076 [ 8174 ] {} DDLWorker(rdb_test_knhvna7w_3): Error on initialization of rdb_test_knhvna7w_3: Code: 57. DB::Exception: Directory for table data store/d9b/d9b57fc1-bbe7-4c06-9ca0-ab88ff19caaf/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:39.784971 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:39.911343 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:39.952308 [ 10540 ] {4fda7b06-a819-4841-9012-5d1046fdfadd} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38354) (comment: 00060_move_to_prewhere_and_sets.sql) (in query: SELECT uniq(URL) FROM test.hits WHERE TraficSourceID IN (7);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:39.975610 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:39.976093 [ 10540 ] {4fda7b06-a819-4841-9012-5d1046fdfadd} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:40.310572 [ 3098 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:40.353289 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:41.015412 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:41.265804 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:41.313472 [ 3114 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:41.409217 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:41.608167 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:41.815906 [ 10192 ] {9d650804-c794-436f-b056-5795608a4dcc} executeQuery: Code: 159. DB::Exception: Distributed DDL task /test/test_knhvna7w/rdb/log/query-0000000010 is not finished on 3 of 4 hosts (0 of them are currently executing the task, 2 are inactive). They are going to execute the query in background. Was waiting for 5.691952007 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42784) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_knhvna7w.t33 (n int) engine=Log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::DDLQueryStatusSource::generate() @ 0x0000000029b11829 4. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 5. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 10. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:41.820570 [ 10192 ] {9d650804-c794-436f-b056-5795608a4dcc} TCPHandler: Code: 159. DB::Exception: Distributed DDL task /test/test_knhvna7w/rdb/log/query-0000000010 is not finished on 3 of 4 hosts (0 of them are currently executing the task, 2 are inactive). They are going to execute the query in background. Was waiting for 5.691952007 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::DDLQueryStatusSource::generate() @ 0x0000000029b11829 4. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 5. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 10. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:41.837271 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:41.845090 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:41.861131 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:42.269648 [ 10931 ] {4d9d688a-60de-4a0e-aa1b-cdfe78243d24} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38402) (comment: 00158_cache_dictionary_has.sql) (in query: SELECT count() from test.hits PREWHERE WatchID % 1400 == 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:306: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349fbb 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:42.274367 [ 10931 ] {4d9d688a-60de-4a0e-aa1b-cdfe78243d24} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=13, jobs=13). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:306: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349fbb 8. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:42.316946 [ 3143 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:42.634561 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:42.755343 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:42.841954 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:43.321400 [ 3185 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:43.333296 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:43.390321 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:43.415635 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:43.543303 [ 5075 ] {} DDLWorker(rdb_test_knhvna7w_2): Error on initialization of rdb_test_knhvna7w_2: Code: 57. DB::Exception: Directory for table data store/062/062c51b9-e437-4a7b-aa75-0ec0f2a04d1a/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:43.750344 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:44.126130 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:44.270041 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:44.324935 [ 3190 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:44.350280 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:44.658184 [ 10540 ] {4254668d-41aa-441e-b692-cf754608ba6d} executeQuery: Code: 264. DB::Exception: Table test.join (96165169-ef74-434f-8355-4287a5cc46ec) needs the same join_use_nulls setting as present in LEFT or FULL JOIN. (INCOMPATIBLE_TYPE_OF_JOIN) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38494) (comment: 00065_loyalty_with_storage_join.sql) (in query: SELECT loyalty, count() FROM hits SEMI LEFT JOIN join USING UserID GROUP BY loyalty ORDER BY loyalty ASC;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Storages/StorageJoin.cpp:239: DB::StorageJoin::getJoinLocked(std::shared_ptr, std::shared_ptr, std::vector> const&) const @ 0x000000002b82f5e2 5. ./build_docker/./src/Planner/PlannerJoins.cpp:0: DB::chooseJoinAlgorithm(std::shared_ptr&, std::shared_ptr const&, DB::Block const&, DB::Block const&, std::shared_ptr const&) @ 0x00000000294a520e 6. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1490: DB::(anonymous namespace)::buildQueryPlanForJoinNode(std::shared_ptr const&, DB::JoinTreeQueryPlan, DB::JoinTreeQueryPlan, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002947642f 7. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::buildJoinTreeQueryPlan(std::shared_ptr const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002945f174 8. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000002942c986 9. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 10. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 14. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 18. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:44.662450 [ 10540 ] {4254668d-41aa-441e-b692-cf754608ba6d} TCPHandler: Code: 264. DB::Exception: Table test.join (96165169-ef74-434f-8355-4287a5cc46ec) needs the same join_use_nulls setting as present in LEFT or FULL JOIN. (INCOMPATIBLE_TYPE_OF_JOIN), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Storages/StorageJoin.cpp:239: DB::StorageJoin::getJoinLocked(std::shared_ptr, std::shared_ptr, std::vector> const&) const @ 0x000000002b82f5e2 5. ./build_docker/./src/Planner/PlannerJoins.cpp:0: DB::chooseJoinAlgorithm(std::shared_ptr&, std::shared_ptr const&, DB::Block const&, DB::Block const&, std::shared_ptr const&) @ 0x00000000294a520e 6. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1490: DB::(anonymous namespace)::buildQueryPlanForJoinNode(std::shared_ptr const&, DB::JoinTreeQueryPlan, DB::JoinTreeQueryPlan, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002947642f 7. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::buildJoinTreeQueryPlan(std::shared_ptr const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002945f174 8. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000002942c986 9. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 10. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 14. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 18. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:44.739409 [ 8174 ] {} DDLWorker(rdb_test_knhvna7w_3): Error on initialization of rdb_test_knhvna7w_3: Code: 57. DB::Exception: Directory for table data store/b86/b86d7be2-f6f9-4060-9ef0-f6451da6df11/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:44.880338 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:45.178172 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:45.209512 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:45.328012 [ 3240 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:46.044262 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:46.296351 [ 10539 ] {04b247f8-13d3-4e37-99ff-124ee59e6a03} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function multiSearchAny: In scope SELECT multiSearchAny(materialize('Hello World'), materialize([])). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38478) (comment: 02294_stringsearch_with_nonconst_needle.sql) (in query: select multiSearchAny(materialize('Hello World'), materialize([]));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::FunctionsMultiStringSearch>::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x0000000015c3d8c9 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:46.298817 [ 10539 ] {04b247f8-13d3-4e37-99ff-124ee59e6a03} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function multiSearchAny: In scope SELECT multiSearchAny(materialize('Hello World'), materialize([])). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::FunctionsMultiStringSearch>::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x0000000015c3d8c9 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:46.330834 [ 3260 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:46.343003 [ 10544 ] {11c2b6ad-bd21-4b5e-a603-131be969abcc} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=769, jobs=735). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38504) (comment: 00091_prewhere_two_conditions.sql) (in query: SELECT uniq(URL) FROM test.hits PREWHERE toTimeZone(EventTime, 'Asia/Dubai') >= '2014-03-20 00:00:00' AND URL != '' AND toTimeZone(EventTime, 'Asia/Dubai') < '2014-03-21 00:00:00';), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:46.344715 [ 10544 ] {11c2b6ad-bd21-4b5e-a603-131be969abcc} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=769, jobs=735). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:46.428899 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:46.465874 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:46.801269 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:46.821215 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:47.040418 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:47.208602 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:47.314338 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:47.333482 [ 2781 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:47.334495 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:47.655475 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:47.723865 [ 10539 ] {65e9414d-15ad-469d-8ccc-1de487586ad4} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function multiSearchFirstIndex: In scope SELECT multiSearchFirstIndex(materialize('Hello World'), materialize([])). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38478) (comment: 02294_stringsearch_with_nonconst_needle.sql) (in query: select multiSearchFirstIndex(materialize('Hello World'), materialize([]));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::FunctionsMultiStringSearch>::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x0000000015c50f09 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:47.728739 [ 10539 ] {65e9414d-15ad-469d-8ccc-1de487586ad4} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function multiSearchFirstIndex: In scope SELECT multiSearchFirstIndex(materialize('Hello World'), materialize([])). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::FunctionsMultiStringSearch>::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x0000000015c50f09 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:48.140829 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:48.177028 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:48.345702 [ 2790 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:48.476055 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:48.667828 [ 5075 ] {} DDLWorker(rdb_test_knhvna7w_2): Error on initialization of rdb_test_knhvna7w_2: Code: 57. DB::Exception: Directory for table data store/b86/b86d7be2-f6f9-4060-9ef0-f6451da6df11/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:49.106917 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:49.133251 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:49.279262 [ 3792 ] {} DDLWorker(rdb_test_knhvna7w_9): Error on initialization of rdb_test_knhvna7w_9: Code: 57. DB::Exception: Directory for table data store/d9b/d9b57fc1-bbe7-4c06-9ca0-ab88ff19caaf/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:49.348195 [ 2807 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:49.438580 [ 10539 ] {f82e118c-8f2d-4807-a692-6e0907f46937} executeQuery: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function multiSearchFirstPosition: In scope SELECT multiSearchFirstPosition(materialize('Hello World'), materialize([])). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38478) (comment: 02294_stringsearch_with_nonconst_needle.sql) (in query: select multiSearchFirstPosition(materialize('Hello World'), materialize([]));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::FunctionsMultiStringSearch>::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x0000000015c67789 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:49.449528 [ 10539 ] {f82e118c-8f2d-4807-a692-6e0907f46937} TCPHandler: Code: 43. DB::Exception: Illegal type Array(Nothing) of argument of function multiSearchFirstPosition: In scope SELECT multiSearchFirstPosition(materialize('Hello World'), materialize([])). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. DB::FunctionsMultiStringSearch>::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x0000000015c67789 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:49.478509 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:49.724797 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:49.851951 [ 8174 ] {} DDLWorker(rdb_test_knhvna7w_3): Error on initialization of rdb_test_knhvna7w_3: Code: 57. DB::Exception: Directory for table data store/b86/b86d7be2-f6f9-4060-9ef0-f6451da6df11/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:49.872848 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:49.930582 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:50.341421 [ 10342 ] {7733b988-45c8-40f4-b5df-cb0839d565b9} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=769, jobs=735). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50658) (comment: 00083_array_filter.sql) (in query: SELECT sum(length(ParsedParams.ValueDouble)) FROM test.hits WHERE notEmpty(ParsedParams.ValueDouble);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:50.346738 [ 10342 ] {7733b988-45c8-40f4-b5df-cb0839d565b9} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=769, jobs=735). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:50.351060 [ 2841 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:50.689159 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:50.787756 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:50.834844 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:51.095216 [ 10544 ] {c5b70bb8-98f6-4a18-9c0a-f818fc706f3d} executeQuery: Code: 452. DB::Exception: Setting allow_experimental_object_type should not be changed. (SETTING_CONSTRAINT_VIOLATION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50674) (comment: 01825_type_json_wide_parts_merge.sql) (in query: SET allow_experimental_object_type = 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges const&, DB::SettingSource) const @ 0x0000000026f807e3 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraints(DB::SettingsChanges const&, DB::SettingSource) @ 0x000000002853b6ad 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::execute() @ 0x00000000294f603b 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:51.107542 [ 10544 ] {c5b70bb8-98f6-4a18-9c0a-f818fc706f3d} TCPHandler: Code: 452. DB::Exception: Setting allow_experimental_object_type should not be changed. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges const&, DB::SettingSource) const @ 0x0000000026f807e3 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraints(DB::SettingsChanges const&, DB::SettingSource) @ 0x000000002853b6ad 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::execute() @ 0x00000000294f603b 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:51.162872 [ 10342 ] {19da5ae9-515d-4b98-9613-c45258509568} executeQuery: Code: 57. DB::Exception: Table test.hits_1m already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50662) (comment: 00157_cache_dictionary.sql) (in query: CREATE TABLE test.hits_1m AS test.hits ENGINE = MergeTree PARTITION BY toYYYYMM(EventDate) ORDER BY (CounterID, EventDate, intHash32(UserID)) SAMPLE BY intHash32(UserID) SETTINGS storage_policy = 'default', index_granularity = 5531, min_bytes_for_wide_part = 0, ratio_of_defaults_for_sparse_serialization = 0.6498346924781799, replace_long_file_name_to_hash = false, max_file_name_length = 128, min_bytes_for_full_part_storage = 230013438, compact_parts_max_bytes_to_buffer = 513191910, compact_parts_max_granules_to_buffer = 1, compact_parts_merge_max_bytes_to_prefetch_part = 27733693, merge_max_block_size = 21328, old_parts_lifetime = 10., prefer_fetch_merged_part_size_threshold = 2762658511, vertical_merge_algorithm_min_rows_to_activate = 494342, vertical_merge_algorithm_min_columns_to_activate = 1, min_merge_bytes_to_use_direct_io = 10737418240, index_granularity_bytes = 24794217, concurrent_part_removal_threshold = 22, allow_vertical_merges_from_compact_to_wide_parts = false, cache_populated_by_fetch = false, marks_compress_block_size = 79363, primary_key_compress_block_size = 33771), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:51.188599 [ 10342 ] {19da5ae9-515d-4b98-9613-c45258509568} TCPHandler: Code: 57. DB::Exception: Table test.hits_1m already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:51.354188 [ 2876 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:51.392929 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:51.499002 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:51.663174 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:51.934227 [ 10342 ] {e3b92aaa-70bc-4f84-ad79-577a5a3127cd} executeQuery: Code: 57. DB::Exception: Table test_3.t already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50694) (comment: 01710_normal_projection_with_query_plan_optimization.sql) (in query: CREATE TABLE t (`id` UInt64, `id2` UInt64, `id3` UInt64, PROJECTION t_reverse (SELECT id, id2, id3 ORDER BY id2, id, id3)) ENGINE = MergeTree ORDER BY id SETTINGS index_granularity = 4, min_bytes_for_wide_part = 1038623906, ratio_of_defaults_for_sparse_serialization = 1., replace_long_file_name_to_hash = true, max_file_name_length = 128, min_bytes_for_full_part_storage = 0, compact_parts_max_bytes_to_buffer = 400080538, compact_parts_max_granules_to_buffer = 256, compact_parts_merge_max_bytes_to_prefetch_part = 16221548, merge_max_block_size = 13079, old_parts_lifetime = 134., prefer_fetch_merged_part_size_threshold = 10737418240, vertical_merge_algorithm_min_rows_to_activate = 1, vertical_merge_algorithm_min_columns_to_activate = 100, min_merge_bytes_to_use_direct_io = 3857947961, index_granularity_bytes = 31081340, concurrent_part_removal_threshold = 41, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = false, marks_compress_block_size = 60977, primary_key_compress_block_size = 34762), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:51.936986 [ 10342 ] {e3b92aaa-70bc-4f84-ad79-577a5a3127cd} TCPHandler: Code: 57. DB::Exception: Table test_3.t already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:18:52.356992 [ 2896 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:52.436254 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:52.588032 [ 10544 ] {f41bf6aa-a59a-4d85-923e-2a01333156f3} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=0): (while reading column Goals.EventTime): (while reading from part /var/lib/clickhouse/store/fab/fab61230-e14a-49b2-a373-9edad6fb4784/201403_1_10_1/ in table test.visits (fab61230-e14a-49b2-a373-9edad6fb4784) located on disk default of type local, from mark 57 with max_rows_to_read = 24472): While executing MergeTreeSelect(pool: ReadPoolParallelReplicas, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50682) (comment: 00023_totals_limit.sql) (in query: SELECT goals_alias.ID AS `ym:s:goalDimension`, uniqIf(UserID, (UserID != 0) AND (`_uniq_Goals` = 1)) FROM test.visits ARRAY JOIN Goals AS goals_alias, arrayEnumerateUniq(Goals.ID) AS `_uniq_Goals` WHERE (CounterID = 842440) GROUP BY `ym:s:goalDimension` WITH TOTALS ORDER BY `ym:s:goalDimension` LIMIT 0, 1 FORMAT JSONCompact;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. ./src/IO/ReadBuffer.h:0: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x000000002775f11b 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::CachedCompressedReadBuffer::nextImpl() @ 0x000000002c7f8e5d 14. DB::ReadBuffer::next() @ 0x000000000b528aeb 15. DB::ReadBuffer::readBig(char*, unsigned long) @ 0x000000000cd74061 16. ./build_docker/./src/DataTypes/Serializations/SerializationNumber.cpp:222: DB::SerializationNumber::deserializeBinaryBulk(DB::IColumn&, DB::ReadBuffer&, unsigned long, double) const @ 0x0000000027b95530 17. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027aa00db 18. ./build_docker/./src/DataTypes/Serializations/SerializationNamed.cpp:0: DB::SerializationNamed::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027b756f8 19. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:0: DB::SerializationArray::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027aba241 20. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:196: DB::MergeTreeReaderWide::readData(DB::NameAndTypePair const&, std::shared_ptr const&, COW::immutable_ptr&, unsigned long, bool, unsigned long, unsigned long, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>&, bool) @ 0x000000002c80211e 21. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:158: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c800e8d 22. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 23. ./src/Storages/MergeTree/MergeTreeRangeReader.h:139: DB::MergeTreeRangeReader::continueReadingChain(DB::MergeTreeRangeReader::ReadResult const&, unsigned long&) @ 0x000000002c7df38d 24. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:984: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbd9d 25. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 26. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:38: DB::MergeTreeThreadSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002de66094 27. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 28. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 29. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 30. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 31. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 2025.09.25 21:18:52.603743 [ 10342 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::ffff:127.0.0.1]:60548, local: [::ffff:127.0.0.2]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:52.608564 [ 10536 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::ffff:127.0.0.1]:37028, local: [::ffff:127.0.0.9]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:52.637472 [ 10544 ] {f41bf6aa-a59a-4d85-923e-2a01333156f3} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=0): (while reading column Goals.EventTime): (while reading from part /var/lib/clickhouse/store/fab/fab61230-e14a-49b2-a373-9edad6fb4784/201403_1_10_1/ in table test.visits (fab61230-e14a-49b2-a373-9edad6fb4784) located on disk default of type local, from mark 57 with max_rows_to_read = 24472): While executing MergeTreeSelect(pool: ReadPoolParallelReplicas, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. ./src/IO/ReadBuffer.h:0: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x000000002775f11b 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::CachedCompressedReadBuffer::nextImpl() @ 0x000000002c7f8e5d 14. DB::ReadBuffer::next() @ 0x000000000b528aeb 15. DB::ReadBuffer::readBig(char*, unsigned long) @ 0x000000000cd74061 16. ./build_docker/./src/DataTypes/Serializations/SerializationNumber.cpp:222: DB::SerializationNumber::deserializeBinaryBulk(DB::IColumn&, DB::ReadBuffer&, unsigned long, double) const @ 0x0000000027b95530 17. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027aa00db 18. ./build_docker/./src/DataTypes/Serializations/SerializationNamed.cpp:0: DB::SerializationNamed::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027b756f8 19. ./build_docker/./src/DataTypes/Serializations/SerializationArray.cpp:0: DB::SerializationArray::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027aba241 20. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:196: DB::MergeTreeReaderWide::readData(DB::NameAndTypePair const&, std::shared_ptr const&, COW::immutable_ptr&, unsigned long, bool, unsigned long, unsigned long, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>&, bool) @ 0x000000002c80211e 21. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:158: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c800e8d 22. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 23. ./src/Storages/MergeTree/MergeTreeRangeReader.h:139: DB::MergeTreeRangeReader::continueReadingChain(DB::MergeTreeRangeReader::ReadResult const&, unsigned long&) @ 0x000000002c7df38d 24. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:984: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbd9d 25. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 26. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:38: DB::MergeTreeThreadSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002de66094 27. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 28. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 29. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 30. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 31. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 2025.09.25 21:18:52.687497 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:52.732021 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:52.786854 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:52.937613 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:53.130873 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:53.171161 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:53.404419 [ 2919 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:53.703397 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:53.796673 [ 5075 ] {} DDLWorker(rdb_test_knhvna7w_2): Error on initialization of rdb_test_knhvna7w_2: Code: 57. DB::Exception: Directory for table data store/d9b/d9b57fc1-bbe7-4c06-9ca0-ab88ff19caaf/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:53.857053 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:53.900441 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:54.239531 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:54.255034 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:54.407105 [ 2826 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:54.631677 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:54.759317 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:54.769276 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:55.018255 [ 8174 ] {} DDLWorker(rdb_test_knhvna7w_3): Error on initialization of rdb_test_knhvna7w_3: Code: 57. DB::Exception: Directory for table data store/b86/b86d7be2-f6f9-4060-9ef0-f6451da6df11/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:55.402516 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:55.411779 [ 2965 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:55.588935 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:55.602041 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:56.344217 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:56.392675 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:56.436315 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:56.444341 [ 2992 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:56.553879 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:56.687032 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:56.704445 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:57.127043 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:57.447085 [ 3020 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:57.710314 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:57.760620 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:58.100879 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:58.383197 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:58.487378 [ 3044 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:58.715701 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:58.745501 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:58.912424 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:58.958416 [ 5075 ] {} DDLWorker(rdb_test_knhvna7w_2): Error on initialization of rdb_test_knhvna7w_2: Code: 57. DB::Exception: Directory for table data store/b86/b86d7be2-f6f9-4060-9ef0-f6451da6df11/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:59.134533 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:59.160329 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:59.363898 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:59.494432 [ 3072 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:59.769360 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:18:59.799514 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:00.295435 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:00.352348 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:00.378409 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:00.497439 [ 3096 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:00.922553 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:00.957116 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:01.168478 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:01.500596 [ 3112 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:01.539342 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:02.042567 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:02.110713 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:02.114728 [ 10546 ] {b07d510a-3823-4311-ba40-f9aaa8ab0b54} DatabaseReplicated (rdb_test_knhvna7w_3): Database rdb_test_knhvna7w_3 exists locally, but its replica does not exist in ZooKeeper (/test/test_knhvna7w/rdb/replicas/s2|r1). Assuming it was dropped, will not try to start it up 2025.09.25 21:19:02.223009 [ 10544 ] {b6587781-3b0c-4fe6-9f5e-3c93b1fc504a} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:45500) (comment: 00081_group_by_without_key_and_totals.sql) (in query: SELECT TraficSourceID AS k, count() AS c FROM test.hits WHERE CounterID = 1704509 GROUP BY k WITH TOTALS ORDER BY k SETTINGS totals_mode = 'after_having_exclusive', max_rows_to_group_by = 100000, group_by_overflow_mode = 'any';), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:02.227297 [ 10544 ] {b6587781-3b0c-4fe6-9f5e-3c93b1fc504a} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:02.507002 [ 3135 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:02.568278 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:03.018408 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:03.056491 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:03.147111 [ 10544 ] {6f79baee-0849-46e7-9dbb-3023c6bfcc0a} executeQuery: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:45536) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_knhvna7w_3.t2 as system.query_log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:03.154692 [ 10544 ] {6f79baee-0849-46e7-9dbb-3023c6bfcc0a} TCPHandler: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:03.290999 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:03.509602 [ 3156 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:03.548503 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:03.558932 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:03.606775 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:03.967491 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:03.990589 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:04.011484 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:04.101611 [ 5075 ] {} DDLWorker(rdb_test_knhvna7w_2): Error on initialization of rdb_test_knhvna7w_2: Code: 57. DB::Exception: Directory for table data store/b86/b86d7be2-f6f9-4060-9ef0-f6451da6df11/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:04.465506 [ 9743 ] {d22d3778-1eed-467f-921d-ac962a9544bc} executeQuery: Code: 452. DB::Exception: Setting allow_experimental_analyzer should not be changed. (SETTING_CONSTRAINT_VIOLATION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:45566) (comment: 00172_early_constant_folding.sql) (in query: EXPLAIN PIPELINE SELECT count(JavaEnable) FROM test.hits WHERE WatchID = 1 OR Title = 'next' OR URL = 'prev' OR URL = '???' OR 1 SETTINGS enable_analyzer = 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges const&, DB::SettingSource) const @ 0x0000000026f807e3 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraints(DB::SettingsChanges const&, DB::SettingSource) @ 0x000000002853b6ad 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::executeForCurrentContext(bool) @ 0x00000000294f6701 9. ./src/Interpreters/InterpreterSetQuery.h:14: DB::applySettingsFromSelectWithUnion(DB::ASTSelectWithUnionQuery const&, std::shared_ptr) @ 0x00000000294f7ccd 10. ./build_docker/./src/Interpreters/InterpreterSetQuery.cpp:65: DB::InterpreterSetQuery::applySettingsFromQuery(std::shared_ptr const&, std::shared_ptr) @ 0x00000000294f6e01 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::applySettingsFromQuery(std::shared_ptr const&, std::shared_ptr) @ 0x00000000294f7439 12. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b33ca4 13. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 15. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 19. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 20. asan_thread_start(void*) @ 0x000000000aa43059 21. ? @ 0x00007f12cd75eac3 22. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:04.486849 [ 9743 ] {d22d3778-1eed-467f-921d-ac962a9544bc} TCPHandler: Code: 452. DB::Exception: Setting allow_experimental_analyzer should not be changed. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges const&, DB::SettingSource) const @ 0x0000000026f807e3 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraints(DB::SettingsChanges const&, DB::SettingSource) @ 0x000000002853b6ad 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::executeForCurrentContext(bool) @ 0x00000000294f6701 9. ./src/Interpreters/InterpreterSetQuery.h:14: DB::applySettingsFromSelectWithUnion(DB::ASTSelectWithUnionQuery const&, std::shared_ptr) @ 0x00000000294f7ccd 10. ./build_docker/./src/Interpreters/InterpreterSetQuery.cpp:65: DB::InterpreterSetQuery::applySettingsFromQuery(std::shared_ptr const&, std::shared_ptr) @ 0x00000000294f6e01 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::applySettingsFromQuery(std::shared_ptr const&, std::shared_ptr) @ 0x00000000294f7439 12. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b33ca4 13. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 15. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 19. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 20. asan_thread_start(void*) @ 0x000000000aa43059 21. ? @ 0x00007f12cd75eac3 22. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:04.513954 [ 3191 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:05.003916 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:05.251876 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:05.289620 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:05.469076 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:05.599022 [ 3223 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:05.695156 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:05.739420 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:06.192575 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:06.218288 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:06.318852 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:06.602847 [ 3239 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:06.849996 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:06.894829 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:07.263936 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:07.606073 [ 2766 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:08.041464 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:08.066209 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:08.327874 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:08.569307 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:08.609116 [ 2750 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:09.013211 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:09.122071 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:09.191392 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:09.518076 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:09.612456 [ 2810 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:09.782614 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:09.829140 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:10.254415 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:10.298668 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:10.392421 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:10.615359 [ 2833 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:10.673691 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:10.755101 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:11.237163 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:11.570012 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:11.618582 [ 2851 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:11.795544 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:11.908932 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:12.539413 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:12.621995 [ 2889 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:12.649582 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:12.704130 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:12.887170 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:12.928953 [ 10546 ] {c8c8ebea-1a48-4cdd-80ef-c500bdabb303} DatabaseReplicated (rdb_test_knhvna7w_2): Database rdb_test_knhvna7w_2 exists locally, but its replica does not exist in ZooKeeper (/test/test_knhvna7w/rdb/replicas/s1|r2). Assuming it was dropped, will not try to start it up 2025.09.25 21:19:12.990465 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:13.030572 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:13.414144 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:13.690905 [ 2915 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:13.768486 [ 10544 ] {1ef1f820-806b-4e1e-8389-8e114f92c4a7} executeQuery: Code: 48. DB::Exception: Truncate is not supported by storage Null. (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:45554) (comment: 00041_aggregating_materialized_view.sql) (in query: DROP TABLE test.visits_null;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:13.773011 [ 10544 ] {1ef1f820-806b-4e1e-8389-8e114f92c4a7} TCPHandler: Code: 48. DB::Exception: Truncate is not supported by storage Null. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:13.944160 [ 11796 ] {35e71110-0870-4137-9607-05b1b22f6a2d} executeQuery: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:44554) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_knhvna7w_2.t2 as system.query_log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:13.958743 [ 11796 ] {35e71110-0870-4137-9607-05b1b22f6a2d} TCPHandler: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:13.961969 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:14.087181 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:14.193979 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:14.493406 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:14.539485 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:14.576153 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:14.694316 [ 2932 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:15.258652 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:15.283533 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:15.352728 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:15.697365 [ 2963 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:16.082372 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:16.362669 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:16.413274 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:16.442238 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:16.632965 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:16.700345 [ 2994 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:17.118017 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:17.246849 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:17.271223 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:17.593233 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:17.792420 [ 3012 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:17.793117 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:17.862444 [ 10544 ] {5bf92615-108b-496e-a840-988a0b38b02a} executeQuery: Code: 241. DB::Exception: Memory tracker (for query): fault injected. Would use 1.94 MiB (attempt to allocate chunk of 1768810 bytes), maximum: 9.31 GiB: (while reading column message): (while reading from part /var/lib/clickhouse/store/022/02260661-95d0-4eca-a73d-2543d99ce5b9/202509_189_214_5/ in table system.text_log (02260661-95d0-4eca-a73d-2543d99ce5b9) located on disk default of type local, from mark 0 with max_rows_to_read = 11382): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (MEMORY_LIMIT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:44588) (comment: 00177_memory_bound_merging.sh) (in query: SELECT COUNT() > 0 FROM system.text_log WHERE query_id IN (SELECT query_id FROM system.query_log WHERE query_id != 'query_id_memory_bound_merging_2423924074' AND initial_query_id = 'query_id_memory_bound_merging_2423924074' AND event_date >= yesterday()) AND event_date >= yesterday() AND message ILIKE '%Reading%ranges in order%'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, char const*&&, char const*&&, String&&, long&, String&&) @ 0x000000001adfc3f5 4. ./build_docker/./src/Common/MemoryTracker.cpp:284: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfafab 5. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfa777 6. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:59: CurrentMemoryTracker::alloc(long) @ 0x000000001ad3289d 7. ./build_docker/./src/Common/Allocator.cpp:0: Allocator::realloc(void*, unsigned long, unsigned long, unsigned long) @ 0x000000001ad2f3ff 8. ./src/IO/BufferWithOwnMemory.h:101: DB::Memory>::resize(unsigned long) @ 0x000000001aef2022 9. ./src/IO/BufferWithOwnMemory.h:80: DB::CompressedReadBufferFromFile::nextImpl() @ 0x0000000027763dd2 10. DB::ReadBuffer::next() @ 0x000000000b528aeb 11. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:159: void DB::deserializeBinarySSE2<1>(DB::PODArray, 63ul, 64ul>&, DB::PODArray, 63ul, 64ul>&, DB::ReadBuffer&, unsigned long) @ 0x0000000027c11390 12. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:259: DB::SerializationString::deserializeBinaryBulk(DB::IColumn&, DB::ReadBuffer&, unsigned long, double) const @ 0x0000000027c0ec00 13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027aa00db 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::readData(unsigned long, COW::immutable_ptr&, unsigned long, DB::MergeTreeReaderStream&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) @ 0x000000002c7b254b 15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.cpp:53: DB::MergeTreeReaderCompactSingleBuffer::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7eb79e 16. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 17. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000002c7e0473 18. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbec3 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:53: DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002dea5374 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:19:17.868594 [ 10544 ] {5bf92615-108b-496e-a840-988a0b38b02a} TCPHandler: Code: 241. DB::Exception: Memory tracker (for query): fault injected. Would use 1.94 MiB (attempt to allocate chunk of 1768810 bytes), maximum: 9.31 GiB: (while reading column message): (while reading from part /var/lib/clickhouse/store/022/02260661-95d0-4eca-a73d-2543d99ce5b9/202509_189_214_5/ in table system.text_log (02260661-95d0-4eca-a73d-2543d99ce5b9) located on disk default of type local, from mark 0 with max_rows_to_read = 11382): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, char const*&&, char const*&&, String&&, long&, String&&) @ 0x000000001adfc3f5 4. ./build_docker/./src/Common/MemoryTracker.cpp:284: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfafab 5. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfa777 6. ./build_docker/./src/Common/CurrentMemoryTracker.cpp:59: CurrentMemoryTracker::alloc(long) @ 0x000000001ad3289d 7. ./build_docker/./src/Common/Allocator.cpp:0: Allocator::realloc(void*, unsigned long, unsigned long, unsigned long) @ 0x000000001ad2f3ff 8. ./src/IO/BufferWithOwnMemory.h:101: DB::Memory>::resize(unsigned long) @ 0x000000001aef2022 9. ./src/IO/BufferWithOwnMemory.h:80: DB::CompressedReadBufferFromFile::nextImpl() @ 0x0000000027763dd2 10. DB::ReadBuffer::next() @ 0x000000000b528aeb 11. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:159: void DB::deserializeBinarySSE2<1>(DB::PODArray, 63ul, 64ul>&, DB::PODArray, 63ul, 64ul>&, DB::ReadBuffer&, unsigned long) @ 0x0000000027c11390 12. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:259: DB::SerializationString::deserializeBinaryBulk(DB::IColumn&, DB::ReadBuffer&, unsigned long, double) const @ 0x0000000027c0ec00 13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027aa00db 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::readData(unsigned long, COW::immutable_ptr&, unsigned long, DB::MergeTreeReaderStream&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) @ 0x000000002c7b254b 15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.cpp:53: DB::MergeTreeReaderCompactSingleBuffer::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7eb79e 16. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 17. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000002c7e0473 18. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbec3 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 20. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:53: DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002dea5374 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:19:17.913349 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:18.033645 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:18.066063 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:18.480431 [ 10544 ] {2f74e5d6-c80f-4ad6-b7d6-7cad15a8ec73} DatabaseReplicated (rdb_test_knhvna7w): Database rdb_test_knhvna7w exists locally, but its replica does not exist in ZooKeeper (/test/test_knhvna7w/rdb/replicas/s1|r1). Assuming it was dropped, will not try to start it up 2025.09.25 21:19:18.513201 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:18.573815 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:18.750007 [ 10544 ] {2f74e5d6-c80f-4ad6-b7d6-7cad15a8ec73} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50962) (comment: 02447_drop_database_replica.sh) (in query: attach database rdb_test_knhvna7w), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseOnDisk::iterateMetadataFiles(std::shared_ptr, std::function const&) const @ 0x0000000027cdb3e5 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000027d13ea8 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::TablesLoader::loadTablesAsync(std::unordered_set, std::hash>, std::equal_to>, std::allocator>>) @ 0x00000000291db317 9. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x000000002918eea7 10. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c33d8 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 14. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 18. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:18.755113 [ 10544 ] {2f74e5d6-c80f-4ad6-b7d6-7cad15a8ec73} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseOnDisk::iterateMetadataFiles(std::shared_ptr, std::function const&) const @ 0x0000000027cdb3e5 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000027d13ea8 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::TablesLoader::loadTablesAsync(std::unordered_set, std::hash>, std::equal_to>, std::allocator>>) @ 0x00000000291db317 9. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x000000002918eea7 10. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c33d8 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 14. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 18. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:18.795643 [ 3051 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:18.954395 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:18.998418 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:19.112451 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:19.559754 [ 10544 ] {b465c10c-cc24-4962-9be2-1cd308c571e7} executeQuery: Code: 81. DB::Exception: Database rdb_test_knhvna7w does not exist. (UNKNOWN_DATABASE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50988) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_knhvna7w.t2 as system.query_log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1471: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ae937 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 7. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 9. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 13. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 14. asan_thread_start(void*) @ 0x000000000aa43059 15. ? @ 0x00007f12cd75eac3 16. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:19.580654 [ 10544 ] {b465c10c-cc24-4962-9be2-1cd308c571e7} TCPHandler: Code: 81. DB::Exception: Database rdb_test_knhvna7w does not exist. (UNKNOWN_DATABASE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1471: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ae937 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 6. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 7. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 9. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 13. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 14. asan_thread_start(void*) @ 0x000000000aa43059 15. ? @ 0x00007f12cd75eac3 16. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:19.799172 [ 3070 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:19.900114 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:20.132101 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:20.160216 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:20.234974 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:20.363931 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:20.699384 [ 10544 ] {8f7024e9-a321-43d9-8159-7f5017e07f1d} executeQuery: Code: 81. DB::Exception: Database rdb_test_knhvna7w does not exist. Maybe you meant rdb_test_knhvna7w_2?. (UNKNOWN_DATABASE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51010) (comment: 02447_drop_database_replica.sh) (in query: show tables from rdb_test_knhvna7w), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:523: DB::DatabaseCatalog::assertDatabaseExists(String const&) const @ 0x00000000286b3418 5. ./build_docker/./src/Interpreters/InterpreterShowTablesQuery.cpp:0: DB::InterpreterShowTablesQuery::getRewrittenQuery() @ 0x0000000029bcd134 6. ./build_docker/./src/Interpreters/InterpreterShowTablesQuery.cpp:227: DB::InterpreterShowTablesQuery::execute() @ 0x0000000029bce366 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:20.701086 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:20.715252 [ 10544 ] {8f7024e9-a321-43d9-8159-7f5017e07f1d} TCPHandler: Code: 81. DB::Exception: Database rdb_test_knhvna7w does not exist. Maybe you meant rdb_test_knhvna7w_2?. (UNKNOWN_DATABASE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:523: DB::DatabaseCatalog::assertDatabaseExists(String const&) const @ 0x00000000286b3418 5. ./build_docker/./src/Interpreters/InterpreterShowTablesQuery.cpp:0: DB::InterpreterShowTablesQuery::getRewrittenQuery() @ 0x0000000029bcd134 6. ./build_docker/./src/Interpreters/InterpreterShowTablesQuery.cpp:227: DB::InterpreterShowTablesQuery::execute() @ 0x0000000029bce366 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:20.802570 [ 3107 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:20.839928 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:20.889257 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:21.513584 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:21.805827 [ 3132 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:21.989450 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:22.021281 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:22.106530 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:22.443149 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:22.462949 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:22.485649 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:22.808446 [ 3165 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:22.854446 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:22.931131 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:22.954960 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:23.210019 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:23.811468 [ 3196 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:23.951116 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:24.090282 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:24.127823 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:24.489870 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:24.814677 [ 3206 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:25.127596 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:25.289513 [ 10544 ] {c519cf5b-ccbf-458b-9ed0-5ceeda2bca87} executeQuery: Code: 81. DB::Exception: Database rdb_test_knhvna7w does not exist. Maybe you meant rdb_test_knhvna7w_2?. (UNKNOWN_DATABASE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51110) (comment: 02447_drop_database_replica.sh) (in query: drop database rdb_test_knhvna7w), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:664: DB::DatabaseCatalog::getDatabase(String const&) const @ 0x00000000286b7273 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f40bc 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:25.305858 [ 10544 ] {c519cf5b-ccbf-458b-9ed0-5ceeda2bca87} TCPHandler: Code: 81. DB::Exception: Database rdb_test_knhvna7w does not exist. Maybe you meant rdb_test_knhvna7w_2?. (UNKNOWN_DATABASE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:664: DB::DatabaseCatalog::getDatabase(String const&) const @ 0x00000000286b7273 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f40bc 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:25.366192 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:25.404234 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:25.460180 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:25.818653 [ 3237 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:26.133873 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:26.303803 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:26.514796 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:26.663405 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:26.724793 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:26.824478 [ 2760 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:27.324438 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:27.600402 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:27.630014 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:27.835887 [ 2765 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:28.282566 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:28.403468 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:28.433688 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:28.567383 [ 8396 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:28.839447 [ 2800 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:29.110301 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:29.131648 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:29.338379 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:29.845526 [ 2831 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:29.977870 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:30.120836 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:30.159125 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:30.841547 [ 12045 ] {8237cae4-985c-4624-869a-ac9bf9b61c6f} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:47100) (comment: 00145_aggregate_functions_statistics.sql) (in query: SELECT stddevPop(ResolutionWidth) FROM (SELECT ResolutionWidth FROM test.hits LIMIT 0);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:30.847045 [ 12045 ] {8237cae4-985c-4624-869a-ac9bf9b61c6f} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:30.848682 [ 2859 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:30.963096 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:31.235546 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:31.280397 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:31.748420 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:31.852366 [ 2884 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:31.979188 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:32.380746 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:32.426145 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:32.466799 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:32.855709 [ 2906 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:32.894650 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:32.897063 [ 11796 ] {49f2fcf3-dd70-40b5-a464-f5fb4ba50aa5} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:47174) (comment: 02447_drop_database_replica.sh) (in query: drop database rdb_test_knhvna7w_4), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:32.912971 [ 11796 ] {49f2fcf3-dd70-40b5-a464-f5fb4ba50aa5} TCPHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:33.144528 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:33.547240 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:33.600588 [ 11796 ] {54049c3e-4168-440e-aaae-a18eed65ed37} executeQuery: Code: 62. DB::Exception: Cannot TRUNCATE dictionary. (SYNTAX_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:47176) (comment: 00158_cache_dictionary_has.sql) (in query: DROP DICTIONARY IF EXISTS db_dict.cache_hits;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:262: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f1adc 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:33.623152 [ 11796 ] {54049c3e-4168-440e-aaae-a18eed65ed37} TCPHandler: Code: 62. DB::Exception: Cannot TRUNCATE dictionary. (SYNTAX_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:262: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f1adc 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:33.730168 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:33.761264 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:33.858949 [ 2931 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:33.955390 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:34.121340 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:34.168429 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:34.298331 [ 2936 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:34.689002 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:34.863697 [ 2951 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:35.200101 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:35.301648 [ 2963 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:35.374355 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:35.386339 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:35.564697 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:35.729497 [ 11796 ] {ce545be5-ca57-42bc-8593-1e66a43e6378} executeQuery: Code: 57. DB::Exception: Table test_9.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:47214) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:35.738713 [ 11796 ] {ce545be5-ca57-42bc-8593-1e66a43e6378} TCPHandler: Code: 57. DB::Exception: Table test_9.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:35.838690 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:35.866915 [ 2979 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:36.096294 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:36.157204 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:36.177696 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:36.306777 [ 3003 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:36.819889 [ 11796 ] {58a8c0fb-647b-4429-9217-845cfbf95e3f} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:47220) (comment: 00051_min_max_array.sql) (in query: SELECT CounterID, count(), max(GoalsReached), min(GoalsReached), minIf(GoalsReached, notEmpty(GoalsReached)) FROM test.hits GROUP BY CounterID ORDER BY count() DESC LIMIT 20 SETTINGS optimize_aggregation_in_order = 1 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:36.870349 [ 3022 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:36.898988 [ 11796 ] {58a8c0fb-647b-4429-9217-845cfbf95e3f} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:36.982369 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:37.264413 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:37.309161 [ 3034 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:37.343216 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:37.453027 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:37.702970 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:37.873770 [ 3047 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:38.313232 [ 3060 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:38.476440 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:38.530549 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:38.764213 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:38.876937 [ 3077 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:38.920052 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:38.936395 [ 10546 ] {70aa3515-0c33-4be7-86d3-b9df47e1d278} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38286) (comment: 00097_constexpr_in_index.sql) (in query: -- Even in presence of OR, we evaluate the "0 IN (1, 2, 3)" as a constant expression therefore it does not prevent the index analysis. SELECT count() FROM test.hits WHERE CounterID IN (14917930, 33034174) OR 0 IN (1, 2, 3) SETTINGS max_rows_to_read = 1000000, force_primary_key = 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:38.949239 [ 10546 ] {70aa3515-0c33-4be7-86d3-b9df47e1d278} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:39.275875 [ 11796 ] {d42b28d8-4bbf-41e2-b825-455c24388ab3} executeQuery: Code: 57. DB::Exception: Table test_9.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38296) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:39.287010 [ 11796 ] {d42b28d8-4bbf-41e2-b825-455c24388ab3} TCPHandler: Code: 57. DB::Exception: Table test_9.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:39.317020 [ 3088 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:39.449291 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:39.815415 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:39.865433 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:39.879958 [ 3107 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:40.289890 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:40.320491 [ 3117 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:40.883314 [ 3132 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:40.950398 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:41.019422 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:41.030463 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:41.323385 [ 3154 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:41.335490 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:41.390590 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:41.518487 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:41.886660 [ 3161 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:42.419287 [ 3163 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:42.591622 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:42.595733 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:42.639651 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:42.889905 [ 3194 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:43.172818 [ 10546 ] {c39cd753-8871-4e07-ae81-05d17c37d9b6} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_01iwczf0, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38392) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from database rdb_test_01iwczf0), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./src/Databases/DatabaseReplicated.h:76: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab2fb 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:43.183932 [ 10546 ] {c39cd753-8871-4e07-ae81-05d17c37d9b6} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_01iwczf0, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./src/Databases/DatabaseReplicated.h:76: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab2fb 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:43.265025 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:43.473590 [ 3179 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:43.499404 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:43.552333 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:43.893212 [ 3206 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:44.129450 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:44.151379 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:44.301447 [ 10546 ] {238c2fba-dff3-49ea-951a-969b9dd85fdb} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_01iwczf0, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38442) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 'r1' from shard 's1' from database rdb_test_01iwczf0), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./src/Databases/DatabaseReplicated.h:76: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab2fb 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:44.305354 [ 10546 ] {238c2fba-dff3-49ea-951a-969b9dd85fdb} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_01iwczf0, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./src/Databases/DatabaseReplicated.h:76: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab2fb 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:44.315003 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:44.477110 [ 3237 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:44.684324 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:44.771716 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:44.896397 [ 3234 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:45.189528 [ 10546 ] {75e5bfac-7777-4dd2-aae1-73583c7eb1e9} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_01iwczf0, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38464) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from zkpath '/test/test_01iwczf0/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:45.195446 [ 10546 ] {75e5bfac-7777-4dd2-aae1-73583c7eb1e9} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_01iwczf0, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:45.277655 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:45.480133 [ 3259 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:45.496055 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:45.530828 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:45.848993 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:45.898643 [ 2767 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:46.172158 [ 10546 ] {d4fff300-8553-49c1-8963-f108b31c1805} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_01iwczf0, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38474) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from zkpath '/test/test_01iwczf0/rdb/'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:46.194936 [ 10546 ] {d4fff300-8553-49c1-8963-f108b31c1805} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_01iwczf0, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:46.476739 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:46.479297 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:46.483394 [ 2785 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:46.537072 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:46.901140 [ 2790 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:47.042178 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:47.095894 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:47.124170 [ 10546 ] {79d89c3f-b8e9-4651-b157-8931d977a697} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_01iwczf0, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48026) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 'r1' from shard 's1' from zkpath '/test/test_01iwczf0/rdb/'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:47.151362 [ 10546 ] {79d89c3f-b8e9-4651-b157-8931d977a697} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_01iwczf0, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:47.281447 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:47.486165 [ 2817 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:47.571843 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:47.620731 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:47.817974 [ 10546 ] {7a3083f1-24ed-47d9-8b2f-2aba778b849d} executeQuery: Code: 36. DB::Exception: Path /test/test_01iwczf0 does not look like a path of Replicated database. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48054) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from zkpath '/test/test_01iwczf0/'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1501: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cbfc 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:47.824186 [ 10546 ] {7a3083f1-24ed-47d9-8b2f-2aba778b849d} TCPHandler: Code: 36. DB::Exception: Path /test/test_01iwczf0 does not look like a path of Replicated database. (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1501: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cbfc 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:47.903289 [ 2819 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:48.027522 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:48.497160 [ 2836 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:48.543048 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:48.586078 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:48.653971 [ 10546 ] {fe6d1a4a-10cd-4dbd-94f2-e94818b21c95} executeQuery: Code: 36. DB::Exception: Replica s2|r1 does not exist (database path: /test/test_01iwczf0/rdb). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48068) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's2|r1' from zkpath '/test/test_01iwczf0/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:48.677181 [ 10546 ] {fe6d1a4a-10cd-4dbd-94f2-e94818b21c95} TCPHandler: Code: 36. DB::Exception: Replica s2|r1 does not exist (database path: /test/test_01iwczf0/rdb). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:48.903694 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:48.905953 [ 2850 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:49.099347 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:49.141923 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:49.454688 [ 10546 ] {d3193a79-b876-45ac-9481-1cec07676d5e} executeQuery: Code: 36. DB::Exception: Replica r1|s1 does not exist (database path: /test/test_01iwczf0/rdb). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48094) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1' from shard 'r1' from zkpath '/test/test_01iwczf0/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:49.479024 [ 10546 ] {d3193a79-b876-45ac-9481-1cec07676d5e} TCPHandler: Code: 36. DB::Exception: Replica r1|s1 does not exist (database path: /test/test_01iwczf0/rdb). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:49.500003 [ 2847 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:49.737035 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:49.798577 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:49.908628 [ 2870 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:49.998435 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:50.194356 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:50.231480 [ 10546 ] {d0ae33fe-31bd-4150-b569-476fd4b79571} executeQuery: Code: 36. DB::Exception: Replica s1|s1|r1 does not exist (database path: /test/test_01iwczf0/rdb). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48120) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from shard 's1' from zkpath '/test/test_01iwczf0/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:50.240650 [ 10546 ] {d0ae33fe-31bd-4150-b569-476fd4b79571} TCPHandler: Code: 36. DB::Exception: Replica s1|s1|r1 does not exist (database path: /test/test_01iwczf0/rdb). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:50.259998 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:50.503265 [ 2888 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:50.911558 [ 2903 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:50.926049 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:51.012429 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:51.034314 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:51.098472 [ 10546 ] {802d4f90-8906-4ed7-b218-c593b62a04e7} executeQuery: Code: 36. DB::Exception: Invalid replica name, '/' is not allowed: s2/r1. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48150) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's2/r1' from zkpath '/test/test_01iwczf0/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x000000000aafc574 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1492: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cca3 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:51.112543 [ 10546 ] {802d4f90-8906-4ed7-b218-c593b62a04e7} TCPHandler: Code: 36. DB::Exception: Invalid replica name, '/' is not allowed: s2/r1. (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x000000000aafc574 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1492: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cca3 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:51.264102 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:51.463404 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:51.506094 [ 2907 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:51.581937 [ 10546 ] {93e2dd35-4a6f-4932-98f2-f351c15200d4} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=735). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48154) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: SELECT * FROM parsing_with_names;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:51.593484 [ 10546 ] {93e2dd35-4a6f-4932-98f2-f351c15200d4} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=735). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:51.780810 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:51.914611 [ 2924 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:51.976195 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:51.993330 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:52.172985 [ 5435 ] {} DDLWorker(rdb_test_01iwczf0_2): Error on initialization of rdb_test_01iwczf0_2: Code: 57. DB::Exception: Directory for table data store/45e/45e5676b-65fe-4f96-9bfb-32419c19df37/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:52.322537 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:52.508934 [ 2954 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:52.869350 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:52.890590 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:52.917356 [ 2957 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:53.197807 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:53.390644 [ 3493 ] {} DDLWorker(rdb_test_01iwczf0_3): Error on initialization of rdb_test_01iwczf0_3: Code: 57. DB::Exception: Directory for table data store/45e/45e5676b-65fe-4f96-9bfb-32419c19df37/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:53.474385 [ 10342 ] {cbb36141-d01d-485f-a74e-a51854841a8f} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 60.160565535 seconds, maximum: 60. (TIMEOUT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50702) (comment: 01290_max_execution_speed_distributed.sql) (in query: SELECT count('special query for 01290_max_execution_speed_distributed') FROM remote('127.0.0.{2,3}', numbers(1000000));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002785801a 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000027856784 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:53.510873 [ 10537 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::ffff:127.0.0.1]:39682, local: [::ffff:127.0.0.3]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:53.512934 [ 2974 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:53.515774 [ 10546 ] {b5dd4df3-16d8-4808-ad86-143b6cccb8fd} executeQuery: Code: 57. DB::Exception: Table test_9.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48180) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:53.522959 [ 10546 ] {b5dd4df3-16d8-4808-ad86-143b6cccb8fd} TCPHandler: Code: 57. DB::Exception: Table test_9.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:53.530835 [ 10342 ] {cbb36141-d01d-485f-a74e-a51854841a8f} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 60.160565535 seconds, maximum: 60. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002785801a 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000027856784 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:53.531720 [ 10192 ] {1a5671e7-79df-4f2b-9974-4ac172390dd4} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 60.005478207 seconds, maximum: 60. (TIMEOUT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:60564) (comment: 01290_max_execution_speed_distributed.sql) (in query: SELECT count() AS `count('special query for 01290_max_execution_speed_distributed')` FROM numbers(1000000) AS `__table1`), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002785801a 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000027856784 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:53.558399 [ 10192 ] {1a5671e7-79df-4f2b-9974-4ac172390dd4} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::ffff:127.0.0.2]:9000 -> [::ffff:127.0.0.1]:60564). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001b0bab02 4. ./build_docker/./src/IO/WriteBufferFromPocoSocketChunked.cpp:201: DB::WriteBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf09bf7 5. ./src/IO/WriteBuffer.h:66: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000002d1a9beb 6. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x000000002d18db59 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d176902 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:53.564708 [ 10192 ] {1a5671e7-79df-4f2b-9974-4ac172390dd4} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 60.005478207 seconds, maximum: 60. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002785801a 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000027856784 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:53.576916 [ 10192 ] {1a5671e7-79df-4f2b-9974-4ac172390dd4} TCPHandler: Client has gone away. 2025.09.25 21:19:53.886711 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:53.920833 [ 2986 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:53.975289 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:54.069067 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:54.516662 [ 2996 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:54.923355 [ 3005 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:54.968436 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:55.007510 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:55.092934 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:55.502617 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:55.520534 [ 3022 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:56.000307 [ 3023 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:56.124936 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:56.250609 [ 10546 ] {0ac73073-1e61-4b06-bc0b-cfd882b9e14c} executeQuery: Code: 36. DB::Exception: Replica s1|r1 is active, cannot drop it (database path: /test/test_01iwczf0/rdb). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48226) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from database rdb_test_01iwczf0_2), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1513: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cde0 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab31e 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:56.267356 [ 10546 ] {0ac73073-1e61-4b06-bc0b-cfd882b9e14c} TCPHandler: Code: 36. DB::Exception: Replica s1|r1 is active, cannot drop it (database path: /test/test_01iwczf0/rdb). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1513: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cde0 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab31e 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:56.328541 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:56.423209 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:56.523835 [ 3046 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:56.982309 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:57.005793 [ 3058 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:57.169332 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:57.201802 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:57.386857 [ 5435 ] {} DDLWorker(rdb_test_01iwczf0_2): Error on initialization of rdb_test_01iwczf0_2: Code: 57. DB::Exception: Directory for table data store/45e/45e5676b-65fe-4f96-9bfb-32419c19df37/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:57.526586 [ 3062 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:58.008927 [ 3089 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:58.055847 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:58.073473 [ 8396 ] {90b25eb8-9946-40a2-b416-3e4bcee17304} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48220) (comment: 02892_rocksdb_trivial_count.sql) (in query: -- { echoOn } SELECT count() FROM dict SETTINGS optimize_trivial_approximate_count_query = 0, max_rows_to_read = 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:58.091441 [ 8396 ] {90b25eb8-9946-40a2-b416-3e4bcee17304} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:19:58.311082 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:58.353364 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:58.529331 [ 3092 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:58.586802 [ 3493 ] {} DDLWorker(rdb_test_01iwczf0_3): Error on initialization of rdb_test_01iwczf0_3: Code: 57. DB::Exception: Directory for table data store/fbd/fbd85d96-3653-4e31-a081-e628e3e1fff2/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:58.606495 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:59.011748 [ 3118 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:59.179998 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:59.216544 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:59.232720 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:59.532078 [ 3125 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:19:59.798175 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:00.015297 [ 3139 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:00.170365 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:00.197112 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:00.404555 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:00.534859 [ 3164 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:01.018492 [ 3176 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:01.170232 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:01.196861 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:01.277829 [ 8396 ] {be744beb-8c78-4624-bca6-ea9357cb0c20} executeQuery: Code: 57. DB::Exception: Table test_9.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37288) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:01.297875 [ 8396 ] {be744beb-8c78-4624-bca6-ea9357cb0c20} TCPHandler: Code: 57. DB::Exception: Table test_9.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:01.508371 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:01.510406 [ 5080 ] {} DDLWorker(rdb_test_01iwczf0_9): Error on initialization of rdb_test_01iwczf0_9: Code: 57. DB::Exception: Directory for table data store/d3c/d3cbaf94-bfd7-47e8-a5fd-a29ffe3d3e5c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:01.537847 [ 3196 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:01.631406 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:01.661433 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:01.678848 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:02.021560 [ 3193 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:02.215224 [ 10546 ] {e9c4570b-bcd0-4cd2-8606-80c0025ad78a} executeQuery: Code: 48. DB::Exception: Truncate is not supported by storage Merge. (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37302) (comment: 00022_merge_prewhere.sql) (in query: DROP TABLE test.merge_hits;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:02.219409 [ 10546 ] {e9c4570b-bcd0-4cd2-8606-80c0025ad78a} TCPHandler: Code: 48. DB::Exception: Truncate is not supported by storage Merge. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:02.285416 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:02.526874 [ 5435 ] {} DDLWorker(rdb_test_01iwczf0_2): Error on initialization of rdb_test_01iwczf0_2: Code: 57. DB::Exception: Directory for table data store/d3c/d3cbaf94-bfd7-47e8-a5fd-a29ffe3d3e5c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:02.540902 [ 3216 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:02.645814 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:02.715402 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:02.905017 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:03.024788 [ 3227 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:03.116984 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:03.147687 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:03.544253 [ 3230 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:03.698361 [ 3493 ] {} DDLWorker(rdb_test_01iwczf0_3): Error on initialization of rdb_test_01iwczf0_3: Code: 57. DB::Exception: Directory for table data store/d3c/d3cbaf94-bfd7-47e8-a5fd-a29ffe3d3e5c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:03.856351 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:04.028223 [ 3245 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:04.242341 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:04.305127 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:04.547493 [ 2756 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:04.716727 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:05.031100 [ 2772 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:05.405718 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:05.430222 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:05.555534 [ 2771 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:05.560034 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:05.863516 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:05.893767 [ 12045 ] {b12f24d2-f02c-4e73-b4af-7dac0fa0ebe6} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37388) (comment: 00165_jit_aggregate_functions.sql) (in query: SELECT CounterID, min(WatchID), max(WatchID), sum(WatchID), avg(WatchID), avgWeighted(WatchID, CounterID), count(WatchID), groupBitOr(WatchID), groupBitAnd(WatchID), groupBitXor(WatchID) FROM test.hits GROUP BY CounterID ORDER BY count() DESC LIMIT 20;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:05.998077 [ 12045 ] {b12f24d2-f02c-4e73-b4af-7dac0fa0ebe6} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:06.033913 [ 2758 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:06.251507 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:06.328871 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:06.558511 [ 2812 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:06.749284 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:06.836013 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:06.904439 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:07.037434 [ 2807 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:07.179950 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:07.210816 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:07.224751 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:07.561480 [ 2838 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:07.698979 [ 5435 ] {} DDLWorker(rdb_test_01iwczf0_2): Error on initialization of rdb_test_01iwczf0_2: Code: 57. DB::Exception: Directory for table data store/528/52887b47-c799-45ae-bac6-9fbb533eae81/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:07.858663 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:08.000977 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:08.027169 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:08.044859 [ 2858 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:08.059376 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:08.564637 [ 2873 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:08.803612 [ 3493 ] {} DDLWorker(rdb_test_01iwczf0_3): Error on initialization of rdb_test_01iwczf0_3: Code: 57. DB::Exception: Directory for table data store/d3c/d3cbaf94-bfd7-47e8-a5fd-a29ffe3d3e5c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:08.835145 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:08.901790 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:08.945485 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:09.048347 [ 2880 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:09.571000 [ 2901 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:10.024429 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:10.050894 [ 2886 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:10.211893 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:10.359866 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:10.574273 [ 2907 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:10.613017 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:10.934183 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:11.053660 [ 2928 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:11.127392 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:11.182685 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:11.576498 [ 2944 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:11.621293 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:12.022136 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:12.056209 [ 2938 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:12.067152 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:12.610214 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:12.686126 [ 2960 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:12.805174 [ 5435 ] {} DDLWorker(rdb_test_01iwczf0_2): Error on initialization of rdb_test_01iwczf0_2: Code: 57. DB::Exception: Directory for table data store/d3c/d3cbaf94-bfd7-47e8-a5fd-a29ffe3d3e5c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:12.951599 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:12.978764 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:13.059448 [ 2969 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:13.488618 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:13.689707 [ 2982 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:13.895574 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:13.920406 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:13.937054 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:13.963218 [ 3493 ] {} DDLWorker(rdb_test_01iwczf0_3): Error on initialization of rdb_test_01iwczf0_3: Code: 57. DB::Exception: Directory for table data store/a63/a63e40f9-4925-4214-afda-d3366ab4bc9b/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:14.062997 [ 2992 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:14.222148 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:14.329549 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:14.678095 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:14.693169 [ 3006 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:14.733000 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:15.066877 [ 3022 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:15.076182 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:15.695955 [ 3031 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:15.773597 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:15.807048 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:16.029360 [ 12045 ] {a1b9b5f8-ee8a-4bad-bfd7-4ee1dfe04746} executeQuery: Code: 36. DB::Exception: Unknown geometry type Array(Nothing): In scope SELECT polygonAreaCartesian([]). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43092) (comment: 01308_polygon_area.sql) (in query: SELECT polygonAreaCartesian([]);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. DB::(anonymous namespace)::FunctionPolygonArea>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x00000000172c5c3c 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:20:16.034769 [ 12045 ] {a1b9b5f8-ee8a-4bad-bfd7-4ee1dfe04746} TCPHandler: Code: 36. DB::Exception: Unknown geometry type Array(Nothing): In scope SELECT polygonAreaCartesian([]). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. DB::(anonymous namespace)::FunctionPolygonArea>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x00000000172c5c3c 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:20:16.070209 [ 3051 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:16.150910 [ 10546 ] {236098bd-e8f3-47fe-8386-58ace70854f0} executeQuery: Code: 159. DB::Exception: Distributed DDL task /test/test_01iwczf0/rdb/log/query-0000000009 is not finished on 3 of 4 hosts (0 of them are currently executing the task, 2 are inactive). They are going to execute the query in background. Was waiting for 5.70795203 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43008) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_01iwczf0.t22 (n int) engine=Log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::DDLQueryStatusSource::generate() @ 0x0000000029b11829 4. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 5. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 10. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d3463e7 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:16.155305 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:16.155761 [ 10546 ] {236098bd-e8f3-47fe-8386-58ace70854f0} TCPHandler: Code: 159. DB::Exception: Distributed DDL task /test/test_01iwczf0/rdb/log/query-0000000009 is not finished on 3 of 4 hosts (0 of them are currently executing the task, 2 are inactive). They are going to execute the query in background. Was waiting for 5.70795203 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::DDLQueryStatusSource::generate() @ 0x0000000029b11829 4. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 5. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 10. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d3463e7 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:16.525181 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:16.529248 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:16.556798 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:16.707768 [ 3054 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:17.073595 [ 3067 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:17.244447 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:17.694640 [ 11796 ] {a977770f-d7ee-4bc5-9e30-b65856b86cb6} executeQuery: Code: 63. DB::Exception: Aggregate function with name 'abs' does not exist. In scope SELECT number, abs(number) OVER (PARTITION BY toString(intDiv(number, 3)) ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) FROM numbers(10). (UNKNOWN_AGGREGATE_FUNCTION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:56328) (comment: 01591_window_functions.sql) (in query: -- not a window function select number, abs(number) over (partition by toString(intDiv(number, 3)) rows unbounded preceding) from numbers(10);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&, String&&, String&&) @ 0x000000001bb46720 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3152: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839d7b4 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 8. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 11. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 14. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 15. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 17. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 19. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 23. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 24. asan_thread_start(void*) @ 0x000000000aa43059 25. ? @ 0x00007f12cd75eac3 26. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:17.704124 [ 11796 ] {a977770f-d7ee-4bc5-9e30-b65856b86cb6} TCPHandler: Code: 63. DB::Exception: Aggregate function with name 'abs' does not exist. In scope SELECT number, abs(number) OVER (PARTITION BY toString(intDiv(number, 3)) ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) FROM numbers(10). (UNKNOWN_AGGREGATE_FUNCTION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&, String&&, String&&) @ 0x000000001bb46720 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3152: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839d7b4 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 8. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 11. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 12. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 14. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 15. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 17. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 19. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 23. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 24. asan_thread_start(void*) @ 0x000000000aa43059 25. ? @ 0x00007f12cd75eac3 26. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:17.710629 [ 3090 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:17.744183 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:17.766069 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:17.785755 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:18.044185 [ 5435 ] {} DDLWorker(rdb_test_01iwczf0_2): Error on initialization of rdb_test_01iwczf0_2: Code: 57. DB::Exception: Directory for table data store/d3c/d3cbaf94-bfd7-47e8-a5fd-a29ffe3d3e5c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:18.076666 [ 3101 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:18.128227 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:18.226755 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:18.718196 [ 3108 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:18.979716 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:19.042646 [ 3493 ] {} DDLWorker(rdb_test_01iwczf0_3): Error on initialization of rdb_test_01iwczf0_3: Code: 57. DB::Exception: Directory for table data store/d3c/d3cbaf94-bfd7-47e8-a5fd-a29ffe3d3e5c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:19.079188 [ 3120 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:19.333438 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:19.411210 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:19.655958 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:19.721796 [ 3138 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:19.934424 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:19.959479 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:20.081234 [ 3144 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:20.255424 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:20.723831 [ 3142 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:20.863626 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:21.069845 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:21.083884 [ 3171 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:21.117122 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:21.617517 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:21.636429 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:21.711292 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:21.725768 [ 3097 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:21.935366 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:21.995215 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:22.047120 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:22.086692 [ 3193 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:22.226497 [ 11796 ] {64d7fe2a-369f-4d2d-915e-53d56cfeebfd} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=740). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:56328) (comment: 01591_window_functions.sql) (in query: -- WINDOW clause select 1 window w1 as ();), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:22.228928 [ 12045 ] {f5325d29-0a7c-4f80-b3a1-db9b3b57dfac} executeQuery: Code: 159. DB::Exception: Distributed DDL task /test/test_01iwczf0/rdb/log/query-0000000010 is not finished on 3 of 4 hosts (0 of them are currently executing the task, 2 are inactive). They are going to execute the query in background. Was waiting for 5.023957785 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:56340) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_01iwczf0.t33 (n int) engine=Log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::DDLQueryStatusSource::generate() @ 0x0000000029b11829 4. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 5. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 10. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:22.229721 [ 11796 ] {64d7fe2a-369f-4d2d-915e-53d56cfeebfd} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=740). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:22.229874 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:22.231169 [ 12045 ] {f5325d29-0a7c-4f80-b3a1-db9b3b57dfac} TCPHandler: Code: 159. DB::Exception: Distributed DDL task /test/test_01iwczf0/rdb/log/query-0000000010 is not finished on 3 of 4 hosts (0 of them are currently executing the task, 2 are inactive). They are going to execute the query in background. Was waiting for 5.023957785 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::DDLQueryStatusSource::generate() @ 0x0000000029b11829 4. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 5. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 10. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:22.514688 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:22.565814 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:22.737698 [ 3215 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:23.095375 [ 3228 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:23.203841 [ 5435 ] {} DDLWorker(rdb_test_01iwczf0_2): Error on initialization of rdb_test_01iwczf0_2: Code: 57. DB::Exception: Directory for table data store/3f8/3f81f9d9-8b4f-410e-8295-8de7e0759d23/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:23.310302 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:23.328905 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:23.352866 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:23.595538 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:23.624270 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:23.740456 [ 3247 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:23.793894 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:24.095633 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:24.102328 [ 3258 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:24.112294 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:24.159956 [ 3493 ] {} DDLWorker(rdb_test_01iwczf0_3): Error on initialization of rdb_test_01iwczf0_3: Code: 57. DB::Exception: Directory for table data store/d3c/d3cbaf94-bfd7-47e8-a5fd-a29ffe3d3e5c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:24.486283 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:24.741264 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:24.743728 [ 2768 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:24.753009 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:24.810879 [ 10192 ] {ecacfb95-aa51-46d2-b1cc-63b2eb240b61} executeQuery: Code: 57. DB::Exception: Table test_1.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:56522) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:24.827626 [ 10192 ] {ecacfb95-aa51-46d2-b1cc-63b2eb240b61} TCPHandler: Code: 57. DB::Exception: Table test_1.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:25.105039 [ 2773 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:25.251965 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:25.549629 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:25.580087 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:25.746795 [ 2785 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:26.022698 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:26.108401 [ 2795 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:26.449775 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:26.688460 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:26.735769 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:26.749638 [ 2820 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:27.111513 [ 2821 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:27.263887 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:27.449849 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:27.506183 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:27.752466 [ 2835 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:27.856122 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:27.908088 [ 10192 ] {c0a362f3-f459-4fe1-9a0e-7c9c80d6cad2} executeQuery: Code: 57. DB::Exception: Table test_1.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55120) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:27.919049 [ 10192 ] {c0a362f3-f459-4fe1-9a0e-7c9c80d6cad2} TCPHandler: Code: 57. DB::Exception: Table test_1.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:28.084764 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:28.110748 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:28.114890 [ 2857 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:28.123747 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:28.417661 [ 5435 ] {} DDLWorker(rdb_test_01iwczf0_2): Error on initialization of rdb_test_01iwczf0_2: Code: 57. DB::Exception: Directory for table data store/d3c/d3cbaf94-bfd7-47e8-a5fd-a29ffe3d3e5c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:28.754799 [ 2860 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:28.823759 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:29.118033 [ 2875 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:29.325073 [ 3493 ] {} DDLWorker(rdb_test_01iwczf0_3): Error on initialization of rdb_test_01iwczf0_3: Code: 57. DB::Exception: Directory for table data store/d3c/d3cbaf94-bfd7-47e8-a5fd-a29ffe3d3e5c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:29.329986 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:29.376364 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:29.441918 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:29.467520 [ 11280 ] {} DDLWorker(rdb_test_01iwczf0_9): Error on initialization of rdb_test_01iwczf0_9: Code: 57. DB::Exception: Directory for table data store/3f8/3f81f9d9-8b4f-410e-8295-8de7e0759d23/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:29.758370 [ 2915 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:29.912403 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:30.120324 [ 2900 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:30.598160 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:30.637247 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:30.761955 [ 2934 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:30.894906 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:31.123101 [ 2943 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:31.548993 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:31.588440 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:31.594630 [ 10931 ] {f5268acc-f215-44ff-8afe-c988d23fdba7} executeQuery: Code: 57. DB::Exception: Table test_1.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55212) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:31.600881 [ 10931 ] {f5268acc-f215-44ff-8afe-c988d23fdba7} TCPHandler: Code: 57. DB::Exception: Table test_1.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:31.765475 [ 2961 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:31.769242 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:32.136094 [ 2978 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:32.282920 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:32.342063 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:32.737115 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:32.769735 [ 2988 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:33.178593 [ 2997 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:33.287394 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:33.374942 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:33.395011 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:33.562604 [ 5435 ] {} DDLWorker(rdb_test_01iwczf0_2): Error on initialization of rdb_test_01iwczf0_2: Code: 57. DB::Exception: Directory for table data store/3f8/3f81f9d9-8b4f-410e-8295-8de7e0759d23/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:33.740206 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:33.754989 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:33.776678 [ 3017 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:34.167215 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:34.180492 [ 3012 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:34.200756 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:34.382245 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:34.657667 [ 3493 ] {} DDLWorker(rdb_test_01iwczf0_3): Error on initialization of rdb_test_01iwczf0_3: Code: 57. DB::Exception: Directory for table data store/d3c/d3cbaf94-bfd7-47e8-a5fd-a29ffe3d3e5c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:34.783417 [ 3038 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:35.182998 [ 3045 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:35.386339 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:35.417732 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:35.430328 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:35.745507 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:35.785391 [ 3062 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:36.089808 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:36.131003 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:36.185342 [ 3077 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:36.484358 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:36.843012 [ 3105 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:36.848201 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:36.915249 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:37.188178 [ 3115 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:37.422740 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:37.447827 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:37.453851 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:37.846979 [ 3136 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:38.245683 [ 3146 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:38.438148 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:38.473236 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:38.499126 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:38.591757 [ 11796 ] {36b3d101-31e5-42ac-a6aa-727fa8b80ab0} executeQuery: Code: 57. DB::Exception: Table test_1.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:40652) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:38.600689 [ 11796 ] {36b3d101-31e5-42ac-a6aa-727fa8b80ab0} TCPHandler: Code: 57. DB::Exception: Table test_1.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:38.700371 [ 5435 ] {} DDLWorker(rdb_test_01iwczf0_2): Error on initialization of rdb_test_01iwczf0_2: Code: 57. DB::Exception: Directory for table data store/d3c/d3cbaf94-bfd7-47e8-a5fd-a29ffe3d3e5c/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:38.922275 [ 3161 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:38.968032 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:39.257459 [ 3173 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:39.498020 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:39.630723 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:39.643810 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:39.946871 [ 3194 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:40.260189 [ 3188 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:40.401065 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:40.413705 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:40.438199 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:40.934306 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:40.949065 [ 3214 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:41.242757 [ 10537 ] {2d2d3436-418e-49f6-91f4-c36a391324ce} DatabaseReplicated (rdb_test_01iwczf0_3): Database rdb_test_01iwczf0_3 exists locally, but its replica does not exist in ZooKeeper (/test/test_01iwczf0/rdb/replicas/s2|r1). Assuming it was dropped, will not try to start it up 2025.09.25 21:20:41.264990 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:41.265027 [ 3223 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:41.304654 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:41.727689 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:41.951548 [ 3243 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:41.971514 [ 11796 ] {cd67a98a-cb47-48d4-ba41-f591fec3dc64} executeQuery: Code: 57. DB::Exception: Table test_1.parsing_with_names already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:40694) (comment: 00167_parallel_parsing_with_names_and_types.sh) (in query: CREATE TABLE parsing_with_names(c FixedString(16), a DateTime('Asia/Dubai'), b String) ENGINE=Memory()), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:41.988491 [ 11796 ] {cd67a98a-cb47-48d4-ba41-f591fec3dc64} TCPHandler: Code: 57. DB::Exception: Table test_1.parsing_with_names already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:41.993705 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:42.009215 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:42.096954 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:42.272275 [ 3248 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:42.399213 [ 11796 ] {eb739398-ed05-4f09-b88c-8e9ea830cce0} executeQuery: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:40702) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_01iwczf0_3.t2 as system.query_log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:42.412156 [ 11796 ] {eb739398-ed05-4f09-b88c-8e9ea830cce0} TCPHandler: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:42.470066 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:42.627864 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:42.758230 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:42.954749 [ 2761 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:43.279210 [ 2754 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:43.330086 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:43.872857 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:43.960331 [ 2771 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:43.977582 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:44.275792 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:44.284792 [ 2785 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:44.651173 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:44.743393 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:44.963136 [ 2798 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:45.229891 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:45.287879 [ 2802 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:45.575239 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:45.629275 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:45.832448 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:45.966300 [ 2845 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:46.068577 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:46.268049 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:46.290715 [ 2846 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:46.314663 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:46.767534 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:46.862533 [ 11796 ] {59fb8d56-f0e9-4f3e-9a83-562e767c1561} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:40848) (comment: 00089_position_functions_with_non_constant_arg.sql) (in query: SELECT count() FROM test.hits WHERE positionCaseInsensitive(URL, 'metrika') != positionCaseInsensitive(URL, materialize('metrika'));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:46.871102 [ 11796 ] {59fb8d56-f0e9-4f3e-9a83-562e767c1561} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=14, jobs=14). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:46.969777 [ 2870 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:47.070806 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:47.105368 [ 11796 ] {b844d93f-d62a-44cc-8ebd-493136c84929} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=732). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51448) (comment: 00159_parallel_formatting_tsv_and_friends.sh) (in query: SELECT ClientEventTime::DateTime('Asia/Dubai') as a, MobilePhoneModel as b, ClientIP6 as c FROM test.hits ORDER BY a, b, c LIMIT 3000000 Format TSKV), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:47.112244 [ 11796 ] {b844d93f-d62a-44cc-8ebd-493136c84929} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=732). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:47.116134 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:47.151361 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:47.297761 [ 2890 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:47.385015 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:47.405877 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:47.519342 [ 10342 ] {add0c999-6214-4de5-aae1-2447e2c1983d} executeQuery: Code: 57. DB::Exception: Table test_3.check_query_comment_column already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:40832) (comment: 00725_comment_columns_long.sql) (in query: -- Check `ALTER TABLE table_name COMMENT COLUMN 'comment'` statement with MergeTree engine CREATE TABLE check_query_comment_column ( first_column UInt8 COMMENT 'comment 1', second_column UInt8 COMMENT 'comment 2', third_column UInt8 COMMENT 'comment 3' ) ENGINE = MergeTree() ORDER BY first_column PARTITION BY second_column SAMPLE BY first_column;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:47.535468 [ 10342 ] {add0c999-6214-4de5-aae1-2447e2c1983d} TCPHandler: Code: 57. DB::Exception: Table test_3.check_query_comment_column already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:47.606427 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:47.881734 [ 10192 ] {d734012d-1ed0-4198-887a-86fb853629da} DatabaseReplicated (rdb_test_01iwczf0_2): Database rdb_test_01iwczf0_2 exists locally, but its replica does not exist in ZooKeeper (/test/test_01iwczf0/rdb/replicas/s1|r2). Assuming it was dropped, will not try to start it up 2025.09.25 21:20:47.972808 [ 2915 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:48.020729 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:48.301396 [ 2910 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:48.603727 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:48.655504 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:48.763247 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:48.817812 [ 11796 ] {828fd56b-5582-4949-b33d-0915031c9ed6} executeQuery: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51478) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_01iwczf0_2.t2 as system.query_log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:48.831497 [ 11796 ] {828fd56b-5582-4949-b33d-0915031c9ed6} TCPHandler: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:48.979963 [ 2929 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:49.175500 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:49.284393 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:49.304266 [ 2930 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:49.881052 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:49.897932 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:49.906223 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:49.983404 [ 2948 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:50.307508 [ 2961 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:50.658226 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:50.726670 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:50.754710 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:50.786150 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:50.988938 [ 2984 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:51.127474 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:51.145984 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:51.310506 [ 2977 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:51.433096 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:51.766543 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:51.805939 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:51.992341 [ 3002 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:52.314062 [ 3006 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:52.448176 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:52.995221 [ 3033 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:53.042501 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:53.085866 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:53.151420 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:53.273878 [ 10192 ] {7ea46fe4-8004-4357-83b3-8553acf7044a} DatabaseReplicated (rdb_test_01iwczf0): Database rdb_test_01iwczf0 exists locally, but its replica does not exist in ZooKeeper (/test/test_01iwczf0/rdb/replicas/s1|r1). Assuming it was dropped, will not try to start it up 2025.09.25 21:20:53.317416 [ 3038 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:54.000964 [ 3050 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:54.118337 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:54.145317 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:54.201991 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:54.320259 [ 3067 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:54.447831 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:54.625270 [ 10192 ] {a9049b84-99bb-4cab-baeb-d8ce79589f3c} executeQuery: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51556) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_01iwczf0.t2 as system.query_log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:54.630663 [ 10192 ] {a9049b84-99bb-4cab-baeb-d8ce79589f3c} TCPHandler: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:54.663929 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:54.775432 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:54.819771 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:55.004232 [ 3086 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:55.345358 [ 3099 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:55.660209 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:55.898529 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:55.935468 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:56.007405 [ 3116 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:56.348255 [ 3124 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:56.591941 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:56.622872 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:56.633863 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:56.988244 [ 10192 ] {5870345e-7021-45f5-97e9-0393b25399e7} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51772) (comment: 00146_aggregate_function_uniq.sql) (in query: SELECT RegionID, uniqHLL12(WatchID) AS X FROM remote('127.0.0.{1,2}', test, hits) GROUP BY RegionID HAVING X > 100000 ORDER BY RegionID ASC;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:56.994686 [ 10192 ] {5870345e-7021-45f5-97e9-0393b25399e7} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:57.009783 [ 3144 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:57.072894 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:57.354043 [ 3156 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:57.474000 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:57.491683 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:57.681553 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:57.946890 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:58.011685 [ 3159 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:58.331772 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:58.341949 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:58.356435 [ 3097 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:58.989292 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:59.014544 [ 3192 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:59.148673 [ 10192 ] {05bdee50-f445-44b9-b7ca-a6707288d894} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51804) (comment: 02447_drop_database_replica.sh) (in query: drop database rdb_test_01iwczf0), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000291f4e6a 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 14. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 18. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:59.166550 [ 10192 ] {05bdee50-f445-44b9-b7ca-a6707288d894} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000291f4e6a 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 14. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 18. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 2025.09.25 21:20:59.232847 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:59.262884 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:59.359435 [ 3210 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:59.500736 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:59.679588 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:20:59.723530 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:00.017625 [ 3228 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:00.170979 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:00.362598 [ 3227 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:00.792454 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:00.826289 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:01.020740 [ 3235 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:01.027323 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:01.315101 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:01.365070 [ 2747 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:01.888755 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:01.900916 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:02.023559 [ 2779 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:02.159548 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:02.197427 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:02.246737 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:02.366920 [ 2781 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:02.813132 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:02.921821 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:02.940308 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:03.025618 [ 2810 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:03.290350 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:03.369603 [ 2814 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:03.435969 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:03.460358 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:03.968888 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:04.036450 [ 2830 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:04.278397 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:04.328155 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:04.416854 [ 2835 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:04.507200 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:05.007548 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:05.075050 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:05.185508 [ 2863 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:05.419941 [ 2872 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:05.469751 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:05.774472 [ 10192 ] {b0ea2c3a-0b0b-4a10-a6dc-17e57d80659b} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51858) (comment: 02447_drop_database_replica.sh) (in query: drop database rdb_test_01iwczf0_4), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:05.782416 [ 10192 ] {b0ea2c3a-0b0b-4a10-a6dc-17e57d80659b} TCPHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:05.931764 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:05.986799 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:06.013372 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:06.194981 [ 2893 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:06.422820 [ 2895 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:06.475670 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:06.842024 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:06.909902 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:06.953287 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:07.049729 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:07.198429 [ 2924 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:07.431033 [ 2927 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:07.552325 [ 2932 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:07.823733 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:07.843284 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:08.060678 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:08.201557 [ 2959 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:08.267547 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:08.309085 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:08.311335 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:08.434826 [ 2973 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:08.555816 [ 2972 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:08.657189 [ 10342 ] {ca589b48-93d4-4f42-97d0-7123fd211222} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51542) (comment: 02723_jit_aggregation_bug_48120.sql) (in query: SELECT minIf(num1, num1 >= 5) FROM dummy GROUP BY num2;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:08.669222 [ 10342 ] {ca589b48-93d4-4f42-97d0-7123fd211222} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:08.867198 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:08.929326 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:09.068519 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:09.204384 [ 2992 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:09.437810 [ 2997 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:09.558660 [ 3015 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:09.746760 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:09.759473 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:09.783007 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:10.206398 [ 3026 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:10.440747 [ 3029 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:10.561521 [ 3033 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:10.767038 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:11.026804 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:11.075176 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:11.218841 [ 3052 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:11.444242 [ 3055 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:11.566575 [ 3056 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:11.732926 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:12.149399 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:12.169105 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:12.222400 [ 3077 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:12.447881 [ 3093 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:12.484089 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:12.514528 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:12.570968 [ 3085 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:12.813110 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:13.226057 [ 3104 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:13.304958 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:13.385763 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:13.450505 [ 3116 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:13.573370 [ 3115 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:13.902306 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:13.922353 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:13.935568 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:14.229248 [ 3145 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:14.277720 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:14.453020 [ 3150 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:14.575584 [ 3140 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:14.604036 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:14.645548 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:14.657910 [ 10540 ] {5ecc4185-42d5-44f6-9250-d8c6f14d1a7e} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_t1fje8px, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:52338) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from database rdb_test_t1fje8px), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./src/Databases/DatabaseReplicated.h:76: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab2fb 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:14.668449 [ 10540 ] {5ecc4185-42d5-44f6-9250-d8c6f14d1a7e} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_t1fje8px, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./src/Databases/DatabaseReplicated.h:76: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab2fb 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:14.702033 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:14.891570 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:14.900821 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:14.937649 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:15.235547 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:15.263847 [ 3184 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:15.288133 [ 10540 ] {0fd351c6-1624-4798-ab1a-f98121bd3fd0} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_t1fje8px, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:52348) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 'r1' from shard 's1' from database rdb_test_t1fje8px), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./src/Databases/DatabaseReplicated.h:76: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab2fb 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:15.295525 [ 10540 ] {0fd351c6-1624-4798-ab1a-f98121bd3fd0} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_t1fje8px, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./src/Databases/DatabaseReplicated.h:76: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab2fb 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:15.435447 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:15.455200 [ 3187 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:15.526065 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:15.577959 [ 3196 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:15.706177 [ 12045 ] {3e954497-7a62-417a-8cdd-6f7c442f39c0} executeQuery: Code: 48. DB::Exception: Current join algorithm is supported only for pipelines without totals. (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:56486) (comment: 02516_projections_with_rollup.sql) (in query: -- We are not interested in the result of this query, but it should not produce a logical error. SELECT avg_duration1, avg_duration1 = avg_duration2 FROM ( SELECT sum(bytes), hour, toStartOfHour(datetime) AS hour, avg(duration) AS avg_duration1 FROM video_log GROUP BY hour WITH ROLLUP WITH TOTALS ) LEFT JOIN ( SELECT hour, sum_bytes AS sum_bytes2, avg_duration AS avg_duration2 FROM video_log_result__fuzz_0 ) USING (hour) SETTINGS joined_subquery_requires_alias = 0 FORMAT Null;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/QueryPipeline/QueryPipelineBuilder.cpp:368: DB::QueryPipelineBuilder::joinPipelinesYShaped(std::unique_ptr>, std::unique_ptr>, std::shared_ptr, DB::Block const&, unsigned long, std::vector, std::allocator>>*) @ 0x000000002783e552 5. ./build_docker/./src/Processors/QueryPlan/JoinStep.cpp:60: DB::JoinStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddd4d41 6. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:15.712333 [ 10540 ] {a8439787-120f-4641-966d-d919eed1b7de} executeQuery: Code: 57. DB::Exception: Cannot attach table with UUID ff005b0b-1144-4980-8976-4516917b8a09, because it was detached but still used by some query. Retry later. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:52358) (comment: 01107_atomic_db_detach_attach.sh) (in query: ATTACH TABLE test_01107_test_3.mt), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception, DB::UUIDTag> const&>(int, FormatStringHelperImpl, DB::UUIDTag> const&>::type>, StrongTypedef, DB::UUIDTag> const&) @ 0x0000000027d02954 4. ./build_docker/./src/Databases/DatabaseAtomic.cpp:399: DB::DatabaseAtomic::assertDetachedTableNotInUse(StrongTypedef, DB::UUIDTag> const&) @ 0x0000000027cf2dd4 5. ./build_docker/./src/Databases/DatabaseAtomic.cpp:0: DB::DatabaseAtomic::checkDetachedTableNotInUse(StrongTypedef, DB::UUIDTag> const&) @ 0x0000000027d0163a 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291b8727 7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:15.722318 [ 12045 ] {3e954497-7a62-417a-8cdd-6f7c442f39c0} TCPHandler: Code: 48. DB::Exception: Current join algorithm is supported only for pipelines without totals. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/QueryPipeline/QueryPipelineBuilder.cpp:368: DB::QueryPipelineBuilder::joinPipelinesYShaped(std::unique_ptr>, std::unique_ptr>, std::shared_ptr, DB::Block const&, unsigned long, std::vector, std::allocator>>*) @ 0x000000002783e552 5. ./build_docker/./src/Processors/QueryPlan/JoinStep.cpp:60: DB::JoinStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddd4d41 6. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 7. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:15.726199 [ 10540 ] {a8439787-120f-4641-966d-d919eed1b7de} TCPHandler: Code: 57. DB::Exception: Cannot attach table with UUID ff005b0b-1144-4980-8976-4516917b8a09, because it was detached but still used by some query. Retry later. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception, DB::UUIDTag> const&>(int, FormatStringHelperImpl, DB::UUIDTag> const&>::type>, StrongTypedef, DB::UUIDTag> const&) @ 0x0000000027d02954 4. ./build_docker/./src/Databases/DatabaseAtomic.cpp:399: DB::DatabaseAtomic::assertDetachedTableNotInUse(StrongTypedef, DB::UUIDTag> const&) @ 0x0000000027cf2dd4 5. ./build_docker/./src/Databases/DatabaseAtomic.cpp:0: DB::DatabaseAtomic::checkDetachedTableNotInUse(StrongTypedef, DB::UUIDTag> const&) @ 0x0000000027d0163a 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291b8727 7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:15.907866 [ 13100 ] {01bd2477-5cc4-4e99-93ed-753d9251dfc3} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_t1fje8px, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:52380) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from zkpath '/test/test_t1fje8px/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:15.926072 [ 13100 ] {01bd2477-5cc4-4e99-93ed-753d9251dfc3} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_t1fje8px, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:16.151686 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:16.254520 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:16.256232 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:16.266900 [ 3210 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:16.438232 [ 13109 ] {4f78813a-d1ad-4fe5-bdd1-2fa7a384eef6} executeQuery: Code: 219. DB::Exception: Database test_01107_test_3 cannot be detached, because some tables are still in use. Retry later. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:52396) (comment: 01107_atomic_db_detach_attach.sh) (in query: DETACH DATABASE test_01107_test_3), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Databases/DatabaseAtomic.cpp:441: DB::DatabaseAtomic::assertCanBeDetached(bool) @ 0x0000000027cfb400 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:393: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5572 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:16.459302 [ 3208 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:16.466456 [ 13109 ] {4f78813a-d1ad-4fe5-bdd1-2fa7a384eef6} TCPHandler: Code: 219. DB::Exception: Database test_01107_test_3 cannot be detached, because some tables are still in use. Retry later. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Databases/DatabaseAtomic.cpp:441: DB::DatabaseAtomic::assertCanBeDetached(bool) @ 0x0000000027cfb400 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:393: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5572 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:16.580754 [ 3213 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:16.648208 [ 10540 ] {ad3ba68d-47be-41c0-9904-7247831289fb} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_t1fje8px, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:52400) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from zkpath '/test/test_t1fje8px/rdb/'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:16.660419 [ 10540 ] {ad3ba68d-47be-41c0-9904-7247831289fb} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_t1fje8px, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:16.761401 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:16.940734 [ 3365 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {b21444fd-168e-477d-be6e-80dbd5e17f4b::202509_1_146_29}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:16.966699 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:17.156346 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:17.178837 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:17.224133 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:17.268554 [ 3239 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:17.306202 [ 10546 ] {743e1352-c240-466f-a20b-dd7bf9d9d41c} executeQuery: Code: 305. DB::Exception: There is a local database rdb_test_t1fje8px, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50654) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 'r1' from shard 's1' from zkpath '/test/test_t1fje8px/rdb/'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:17.332006 [ 10546 ] {743e1352-c240-466f-a20b-dd7bf9d9d41c} TCPHandler: Code: 305. DB::Exception: There is a local database rdb_test_t1fje8px, which has the same path in ZooKeeper and the same replica name. Please check the path in query. If you want to drop replica of this database, use `DROP DATABASE`. (TABLE_WAS_NOT_DROPPED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:1066: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&)::$_0::operator()(DB::DatabaseReplicated const*, DB::ASTSystemQuery const&) const @ 0x0000000029bbb90a 5. ./contrib/llvm-project/libcxx/include/__tree:189: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab6fa 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:17.467965 [ 3255 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:17.583197 [ 3257 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:17.908701 [ 10546 ] {5df69b02-b2ae-4580-9efa-6b846ceb7f90} executeQuery: Code: 36. DB::Exception: Path /test/test_t1fje8px does not look like a path of Replicated database. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50674) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from zkpath '/test/test_t1fje8px/'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1501: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cbfc 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:17.949438 [ 10546 ] {5df69b02-b2ae-4580-9efa-6b846ceb7f90} TCPHandler: Code: 36. DB::Exception: Path /test/test_t1fje8px does not look like a path of Replicated database. (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1501: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cbfc 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:18.202322 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:18.270733 [ 2773 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:18.392359 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:18.428782 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:18.470192 [ 2774 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:18.585119 [ 2753 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:18.695861 [ 13109 ] {3228e8a7-4f09-4795-a604-946296a27bda} executeQuery: Code: 36. DB::Exception: Replica s2|r1 does not exist (database path: /test/test_t1fje8px/rdb). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50694) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's2|r1' from zkpath '/test/test_t1fje8px/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:18.710585 [ 13109 ] {3228e8a7-4f09-4795-a604-946296a27bda} TCPHandler: Code: 36. DB::Exception: Replica s2|r1 does not exist (database path: /test/test_t1fje8px/rdb). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:19.026569 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:19.027813 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:19.084743 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:19.272838 [ 2787 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:19.414971 [ 13100 ] {6b1f0437-7814-4f40-8986-ed52994a9e73} executeQuery: Code: 36. DB::Exception: Replica r1|s1 does not exist (database path: /test/test_t1fje8px/rdb). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50720) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1' from shard 'r1' from zkpath '/test/test_t1fje8px/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:19.432028 [ 13100 ] {6b1f0437-7814-4f40-8986-ed52994a9e73} TCPHandler: Code: 36. DB::Exception: Replica r1|s1 does not exist (database path: /test/test_t1fje8px/rdb). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:19.472217 [ 2803 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:19.587656 [ 2796 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:19.896309 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:20.000444 [ 13109 ] {2319f4aa-417c-41ce-af6e-568fcb1cdc7f} executeQuery: Code: 36. DB::Exception: Replica s1|s1|r1 does not exist (database path: /test/test_t1fje8px/rdb). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50750) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from shard 's1' from zkpath '/test/test_t1fje8px/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:20.008766 [ 13109 ] {2319f4aa-417c-41ce-af6e-568fcb1cdc7f} TCPHandler: Code: 36. DB::Exception: Replica s1|s1|r1 does not exist (database path: /test/test_t1fje8px/rdb). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1508: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cd51 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:20.188793 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:20.222786 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:20.275502 [ 2828 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:20.474296 [ 2825 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:20.501376 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:20.589619 [ 2832 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:20.744954 [ 13109 ] {543de928-091d-430c-8a1f-128ef1994fa1} executeQuery: Code: 36. DB::Exception: Invalid replica name, '/' is not allowed: s2/r1. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50774) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's2/r1' from zkpath '/test/test_t1fje8px/rdb'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x000000000aafc574 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1492: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cca3 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:20.753780 [ 13109 ] {543de928-091d-430c-8a1f-128ef1994fa1} TCPHandler: Code: 36. DB::Exception: Invalid replica name, '/' is not allowed: s2/r1. (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x000000000aafc574 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1492: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cca3 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab7d2 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:20.968350 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:20.984307 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:21.277322 [ 2849 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:21.476449 [ 2854 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:21.538554 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:21.557576 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:21.558925 [ 5440 ] {} DDLWorker(rdb_test_t1fje8px_2): Error on initialization of rdb_test_t1fje8px_2: Code: 57. DB::Exception: Directory for table data store/839/839de8f6-08bd-4aa2-a705-5f0454d7b523/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:21.563727 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:21.594894 [ 2867 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:22.019911 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:22.031168 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:22.273798 [ 5082 ] {} DDLWorker(rdb_test_t1fje8px_3): Error on initialization of rdb_test_t1fje8px_3: Code: 57. DB::Exception: Directory for table data store/839/839de8f6-08bd-4aa2-a705-5f0454d7b523/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:22.279828 [ 2887 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:22.479171 [ 2883 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:22.598313 [ 2893 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:22.659468 [ 13100 ] {09fc0d09-1dca-4f4d-9967-ec07fabd736e} executeQuery: Code: 57. DB::Exception: Table test_9.distinct_in_order already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50772) (comment: 00174_distinct_in_order.sql) (in query: CREATE TABLE distinct_in_order (`CounterID` UInt32, `EventDate` Date) ENGINE = MergeTree ORDER BY (CounterID, EventDate) SETTINGS index_granularity = 8192, index_granularity_bytes = '10Mi', min_bytes_for_wide_part = 133124839, ratio_of_defaults_for_sparse_serialization = 0., replace_long_file_name_to_hash = false, max_file_name_length = 128, min_bytes_for_full_part_storage = 364028951, compact_parts_max_bytes_to_buffer = 487275908, compact_parts_max_granules_to_buffer = 133, compact_parts_merge_max_bytes_to_prefetch_part = 18103881, merge_max_block_size = 18966, old_parts_lifetime = 399., prefer_fetch_merged_part_size_threshold = 1, vertical_merge_algorithm_min_rows_to_activate = 1, vertical_merge_algorithm_min_columns_to_activate = 100, min_merge_bytes_to_use_direct_io = 10737418240, concurrent_part_removal_threshold = 0, allow_vertical_merges_from_compact_to_wide_parts = false, cache_populated_by_fetch = true, marks_compress_block_size = 62126, primary_key_compress_block_size = 45190), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:22.662792 [ 13109 ] {b71556fc-e6e2-437c-9be8-93ae868b7597} DatabaseReplicated (test_5): It's not recommended to explicitly specify zookeeper_path and replica_name in ReplicatedMergeTree arguments 2025.09.25 21:21:22.663888 [ 13100 ] {09fc0d09-1dca-4f4d-9967-ec07fabd736e} TCPHandler: Code: 57. DB::Exception: Table test_9.distinct_in_order already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:22.674552 [ 13109 ] {b71556fc-e6e2-437c-9be8-93ae868b7597} executeQuery: Code: 80. DB::Exception: Explicit zookeeper_path and replica_name are specified in ReplicatedMergeTree arguments. If you really want to specify it explicitly, then you should use some macros to distinguish different shards and replicas. (INCORRECT_QUERY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50802) (comment: 02433_default_expression_operator_in.sql) (in query: CREATE TABLE dep ( `id` Int32, `country` LowCardinality(String), `purchase_location` UInt16 MATERIALIZED if(id IN joinGet(concat(currentDatabase(), '.id_join'), 'location', 'CLICK'), 123, 456) ) ENGINE = ReplicatedMergeTree('/test/02433/{database}/dep', '1') ORDER BY tuple();), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:878: DB::DatabaseReplicated::checkTableEngine(DB::ASTCreateQuery const&, DB::ASTStorage&, std::shared_ptr) const @ 0x0000000027d5afc7 5. ./build_docker/./src/Databases/DatabaseReplicated.cpp:957: DB::DatabaseReplicated::checkQueryValid(std::shared_ptr const&, std::shared_ptr) const @ 0x0000000027d5e336 6. ./build_docker/./src/Databases/DatabaseReplicated.cpp:999: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5f0de 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:22.678332 [ 13109 ] {b71556fc-e6e2-437c-9be8-93ae868b7597} TCPHandler: Code: 80. DB::Exception: Explicit zookeeper_path and replica_name are specified in ReplicatedMergeTree arguments. If you really want to specify it explicitly, then you should use some macros to distinguish different shards and replicas. (INCORRECT_QUERY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:878: DB::DatabaseReplicated::checkTableEngine(DB::ASTCreateQuery const&, DB::ASTStorage&, std::shared_ptr) const @ 0x0000000027d5afc7 5. ./build_docker/./src/Databases/DatabaseReplicated.cpp:957: DB::DatabaseReplicated::checkQueryValid(std::shared_ptr const&, std::shared_ptr) const @ 0x0000000027d5e336 6. ./build_docker/./src/Databases/DatabaseReplicated.cpp:999: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5f0de 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:22.685886 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:22.883492 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:22.956341 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:23.046760 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:23.282198 [ 2910 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:23.325252 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:23.481549 [ 2919 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:23.600467 [ 2917 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:24.018918 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:24.046680 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:24.288851 [ 2943 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:24.303128 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:24.483847 [ 2938 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:24.560325 [ 13109 ] {8f6ebb1a-3e2c-4cb9-80d4-830d78117a83} executeQuery: Code: 36. DB::Exception: Replica s1|r1 is active, cannot drop it (database path: /test/test_t1fje8px/rdb). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50872) (comment: 02447_drop_database_replica.sh) (in query: system drop database replica 's1|r1' from database rdb_test_t1fje8px_2), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1513: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cde0 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab31e 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:24.568790 [ 13109 ] {8f6ebb1a-3e2c-4cb9-80d4-830d78117a83} TCPHandler: Code: 36. DB::Exception: Replica s1|r1 is active, cannot drop it (database path: /test/test_t1fje8px/rdb). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&, String const&) @ 0x000000001b1e6bba 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1513: DB::DatabaseReplicated::dropReplica(DB::DatabaseReplicated*, String const&, String const&, String const&, bool) @ 0x0000000027d7cde0 5. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:0: DB::InterpreterSystemQuery::dropDatabaseReplica(DB::ASTSystemQuery&) @ 0x0000000029bab31e 6. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:681: DB::InterpreterSystemQuery::execute() @ 0x0000000029b9a8c1 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:24.595688 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:24.602137 [ 2958 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:24.619128 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:25.194992 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:25.204542 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:25.228701 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:25.291595 [ 2984 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:25.513407 [ 2983 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:25.604724 [ 2977 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:25.630821 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:25.750889 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:26.053022 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:26.294979 [ 3006 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:26.516850 [ 3012 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:26.609458 [ 3025 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:26.709930 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:26.722609 [ 5440 ] {} DDLWorker(rdb_test_t1fje8px_2): Error on initialization of rdb_test_t1fje8px_2: Code: 57. DB::Exception: Directory for table data store/2df/2dffac71-9a20-4f3f-a5f3-1c10d8290423/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:26.729326 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:26.736750 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:27.298424 [ 3047 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:27.474486 [ 5082 ] {} DDLWorker(rdb_test_t1fje8px_3): Error on initialization of rdb_test_t1fje8px_3: Code: 57. DB::Exception: Directory for table data store/2df/2dffac71-9a20-4f3f-a5f3-1c10d8290423/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:27.519723 [ 3065 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:27.611755 [ 3060 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:27.632234 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:27.641450 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:27.717536 [ 13100 ] {ce52513b-8637-4c6e-a121-9c58aab5e08f} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=2): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50838) (comment: 03207_json_read_subcolumns_1_wide_merge_tree.gen.sql) (in query: select distinct arrayJoin(JSONAllPathsWithTypes(json)) as paths_with_types from test order by paths_with_types;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./src/DataTypes/Serializations/ISerialization.h:225: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000027ab5907 12. ./build_docker/./src/DataTypes/Serializations/SerializationObject.cpp:0: DB::SerializationObject::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000027bb87b7 13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:144: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002def1bcc 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 27. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 28. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:21:27.864026 [ 13100 ] {ce52513b-8637-4c6e-a121-9c58aab5e08f} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=2): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./src/DataTypes/Serializations/ISerialization.h:225: DB::SerializationArray::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000027ab5907 12. ./build_docker/./src/DataTypes/Serializations/SerializationObject.cpp:0: DB::SerializationObject::enumerateStreams(DB::ISerialization::EnumerateStreamsSettings&, std::function const&, DB::ISerialization::SubstreamData const&) const @ 0x0000000027bb87b7 13. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 15. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 17. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 18. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 20. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPool.cpp:144: DB::MergeTreeReadPool::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002def1bcc 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 27. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 28. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:21:27.878493 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:27.897372 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:28.043839 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:28.301346 [ 3083 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:28.539845 [ 3087 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:28.617664 [ 3090 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:28.742243 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:28.742323 [ 11278 ] {} DDLWorker(rdb_test_t1fje8px_9): Error on initialization of rdb_test_t1fje8px_9: Code: 57. DB::Exception: Directory for table data store/8e5/8e521274-511c-4053-a286-6b6551a2f792/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:29.013858 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:29.030973 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:29.326708 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:29.348969 [ 3112 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:29.542902 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:29.543232 [ 3117 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:29.620410 [ 3119 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:30.019741 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:30.157982 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:30.216168 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:30.352095 [ 3150 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:30.382929 [ 13109 ] {f230522c-cc53-4af5-b8a8-86f4b70383ab} executeQuery: Code: 420. DB::Exception: Cannot update column `d` with type Dynamic: updates of columns with dynamic subcolumns are not supported. (CANNOT_UPDATE_COLUMN) (version 24.8.14.10508.altinitytest (altinity build)) (from 0.0.0.0:0) (comment: 03226_alter_update_dynamic_json_not_supported.sql) (in query: /* ddl_entry=query-0000000069 */ ALTER TABLE test_5.test UPDATE d = 42 WHERE 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/MutationsInterpreter.cpp:1352: DB::MutationsInterpreter::validate() @ 0x00000000295b40e2 5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:228: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000029174914 6. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x000000002917237f 14. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:30.392711 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:30.486513 [ 13109 ] {f230522c-cc53-4af5-b8a8-86f4b70383ab} executeQuery: Code: 420. DB::Exception: Cannot update column `d` with type Dynamic: updates of columns with dynamic subcolumns are not supported. (CANNOT_UPDATE_COLUMN) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:57524) (comment: 03226_alter_update_dynamic_json_not_supported.sql) (in query: alter table test update d = 42 where 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/MutationsInterpreter.cpp:1352: DB::MutationsInterpreter::validate() @ 0x00000000295b40e2 5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:228: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000029174914 6. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x000000002917237f 14. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:30.489431 [ 13109 ] {f230522c-cc53-4af5-b8a8-86f4b70383ab} TCPHandler: Code: 420. DB::Exception: Cannot update column `d` with type Dynamic: updates of columns with dynamic subcolumns are not supported. (CANNOT_UPDATE_COLUMN), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/MutationsInterpreter.cpp:1352: DB::MutationsInterpreter::validate() @ 0x00000000295b40e2 5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:228: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000029174914 6. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x000000002917237f 14. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:30.546667 [ 3156 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:30.623432 [ 3160 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:30.735697 [ 13109 ] {b4ba675e-2576-4bfc-afd8-1605968f54ca} executeQuery: Code: 420. DB::Exception: Cannot update column `json` with type JSON: updates of columns with dynamic subcolumns are not supported. (CANNOT_UPDATE_COLUMN) (version 24.8.14.10508.altinitytest (altinity build)) (from 0.0.0.0:0) (comment: 03226_alter_update_dynamic_json_not_supported.sql) (in query: /* ddl_entry=query-0000000070 */ ALTER TABLE test_5.test UPDATE json = '{}' WHERE 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/MutationsInterpreter.cpp:1352: DB::MutationsInterpreter::validate() @ 0x00000000295b40e2 5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:228: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000029174914 6. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x000000002917237f 14. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:30.775925 [ 13109 ] {b4ba675e-2576-4bfc-afd8-1605968f54ca} executeQuery: Code: 420. DB::Exception: Cannot update column `json` with type JSON: updates of columns with dynamic subcolumns are not supported. (CANNOT_UPDATE_COLUMN) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:57524) (comment: 03226_alter_update_dynamic_json_not_supported.sql) (in query: alter table test update json = '{}' where 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/MutationsInterpreter.cpp:1352: DB::MutationsInterpreter::validate() @ 0x00000000295b40e2 5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:228: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000029174914 6. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x000000002917237f 14. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:30.779496 [ 13109 ] {b4ba675e-2576-4bfc-afd8-1605968f54ca} TCPHandler: Code: 420. DB::Exception: Cannot update column `json` with type JSON: updates of columns with dynamic subcolumns are not supported. (CANNOT_UPDATE_COLUMN), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/MutationsInterpreter.cpp:1352: DB::MutationsInterpreter::validate() @ 0x00000000295b40e2 5. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:228: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000029174914 6. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x000000002917237f 14. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:31.313015 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:31.354866 [ 3175 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:31.363593 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:31.402892 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:31.549684 [ 3187 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:31.626750 [ 3194 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:31.639189 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:31.974491 [ 5440 ] {} DDLWorker(rdb_test_t1fje8px_2): Error on initialization of rdb_test_t1fje8px_2: Code: 57. DB::Exception: Directory for table data store/8e5/8e521274-511c-4053-a286-6b6551a2f792/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:32.208780 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:32.234400 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:32.358374 [ 3203 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:32.485624 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:32.552508 [ 3190 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:32.553121 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:32.606566 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:32.630067 [ 3220 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:32.641522 [ 5082 ] {} DDLWorker(rdb_test_t1fje8px_3): Error on initialization of rdb_test_t1fje8px_3: Code: 57. DB::Exception: Directory for table data store/8e5/8e521274-511c-4053-a286-6b6551a2f792/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:33.277923 [ 13129 ] {ff28e73e-cb1a-48c1-8614-d163e4f759ad} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:57558) (comment: 00078_group_by_arrays.sql) (in query: SELECT GoalsReached AS k, count() AS c FROM test.hits GROUP BY k ORDER BY c DESC LIMIT 10;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:33.297526 [ 13129 ] {ff28e73e-cb1a-48c1-8614-d163e4f759ad} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=26, jobs=26). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:33.361896 [ 3233 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:33.555895 [ 3243 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:33.629429 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:33.633106 [ 3246 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:33.701014 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:33.796002 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:34.355162 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:34.367081 [ 2756 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:34.569552 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:34.570884 [ 2761 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:34.636517 [ 2762 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:34.652270 [ 13112 ] {c97587f6-6942-4fc5-8cb7-c27581e0fe4a} executeQuery: Code: 43. DB::Exception: Illegal types of arguments (Date, Int64) of function less: In scope SELECT today() < ((2018 - 11) - 14). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:57570) (comment: 00762_date_comparsion.sql) (in query: select today() < 2018-11-14;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionComparison::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x00000000133ca35d 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:34.657989 [ 13112 ] {c97587f6-6942-4fc5-8cb7-c27581e0fe4a} TCPHandler: Code: 43. DB::Exception: Illegal types of arguments (Date, Int64) of function less: In scope SELECT today() < ((2018 - 11) - 14). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionComparison::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x00000000133ca35d 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:35.013605 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:35.055090 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:35.216161 [ 13130 ] {160ff3c1-29dc-4d73-8897-9a8ebefa7184} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:57596) (comment: 01107_atomic_db_detach_attach.sh) (in query: DROP DATABASE test_01107_test_3), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:35.228125 [ 13130 ] {160ff3c1-29dc-4d73-8897-9a8ebefa7184} TCPHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:35.369845 [ 2771 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:35.493969 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:35.512315 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:35.573110 [ 2753 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:35.625908 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:35.638599 [ 2783 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:35.670567 [ 13112 ] {a8365969-4467-4ca1-906d-2be2b304266c} executeQuery: Code: 43. DB::Exception: Illegal types of arguments (Date, UInt8) of function less: In scope SELECT toDate('2018-01-01') < 1. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:57570) (comment: 00762_date_comparsion.sql) (in query: select toDate('2018-01-01') < 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionComparison::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x00000000133ca35d 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:35.673069 [ 13112 ] {a8365969-4467-4ca1-906d-2be2b304266c} TCPHandler: Code: 43. DB::Exception: Illegal types of arguments (Date, UInt8) of function less: In scope SELECT toDate('2018-01-01') < 1. (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionComparison::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x00000000133ca35d 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:36.021627 [ 13112 ] {c8c3bbc7-2838-4b62-b714-967789e41533} executeQuery: Code: 43. DB::Exception: Illegal types of arguments (Date, UInt8) of function equals: In scope SELECT toDate('2018-01-01') = 1. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:57570) (comment: 00762_date_comparsion.sql) (in query: select toDate('2018-01-01') == 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionComparison::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x000000000f31955d 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:36.034627 [ 13112 ] {c8c3bbc7-2838-4b62-b714-967789e41533} TCPHandler: Code: 43. DB::Exception: Illegal types of arguments (Date, UInt8) of function equals: In scope SELECT toDate('2018-01-01') = 1. (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionComparison::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x000000000f31955d 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:36.173663 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:36.190241 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:36.240597 [ 13112 ] {bfc84515-97e5-4e9a-a6d6-5cd3b57e70a2} executeQuery: Code: 43. DB::Exception: Illegal types of arguments (Date, UInt8) of function notEquals: In scope SELECT toDate('2018-01-01') != 1. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:57570) (comment: 00762_date_comparsion.sql) (in query: select toDate('2018-01-01') != 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionComparison::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x00000000161c535d 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:36.241199 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:36.255190 [ 13112 ] {bfc84515-97e5-4e9a-a6d6-5cd3b57e70a2} TCPHandler: Code: 43. DB::Exception: Illegal types of arguments (Date, UInt8) of function notEquals: In scope SELECT toDate('2018-01-01') != 1. (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000ac6ad20 4. DB::FunctionComparison::getReturnTypeImpl(std::vector, std::allocator>> const&) const @ 0x00000000161c535d 5. DB::IFunction::getReturnTypeImpl(std::vector> const&) const @ 0x000000000aa8d2f3 6. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 7. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 8. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 9. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:36.373040 [ 2801 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:36.575158 [ 2817 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:36.640576 [ 2813 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:37.065048 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:37.085062 [ 5440 ] {} DDLWorker(rdb_test_t1fje8px_2): Error on initialization of rdb_test_t1fje8px_2: Code: 57. DB::Exception: Directory for table data store/2df/2dffac71-9a20-4f3f-a5f3-1c10d8290423/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:37.376102 [ 2830 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:37.509913 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:37.524827 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:37.578265 [ 2829 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:37.643689 [ 2836 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:37.767775 [ 5082 ] {} DDLWorker(rdb_test_t1fje8px_3): Error on initialization of rdb_test_t1fje8px_3: Code: 57. DB::Exception: Directory for table data store/2df/2dffac71-9a20-4f3f-a5f3-1c10d8290423/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:38.157883 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:38.379256 [ 2857 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:38.506678 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:38.535566 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:38.581178 [ 2847 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:38.647124 [ 2858 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:38.757781 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:39.218607 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:39.276812 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:39.383280 [ 2889 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:39.509221 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:39.584407 [ 2882 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:39.650277 [ 2888 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:39.785185 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:40.095084 [ 3359 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {847a217e-a513-42bb-858f-ba3e2c92857a::202509_1_95_19}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:40.269015 [ 3360 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {847a217e-a513-42bb-858f-ba3e2c92857a::202509_1_95_19}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:40.276858 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:40.317883 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:40.334841 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:40.385465 [ 2924 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:40.587088 [ 2928 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:40.653301 [ 2934 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:41.051923 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:41.387807 [ 2958 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:41.388152 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:41.590058 [ 2961 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:41.635164 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:41.660069 [ 2965 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:41.680000 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:42.174423 [ 5440 ] {} DDLWorker(rdb_test_t1fje8px_2): Error on initialization of rdb_test_t1fje8px_2: Code: 57. DB::Exception: Directory for table data store/8e5/8e521274-511c-4053-a286-6b6551a2f792/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:42.390136 [ 2988 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:42.404064 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:42.477483 [ 13112 ] {fab7f3ff-20ec-4ceb-a383-216177534bb7} executeQuery: Code: 159. DB::Exception: Distributed DDL task /test/test_t1fje8px/rdb/log/query-0000000009 is not finished on 3 of 4 hosts (0 of them are currently executing the task, 2 are inactive). They are going to execute the query in background. Was waiting for 5.675952418 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:57610) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_t1fje8px.t22 (n int) engine=Log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::DDLQueryStatusSource::generate() @ 0x0000000029b11829 4. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 5. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 10. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d3463e7 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:42.480489 [ 13112 ] {fab7f3ff-20ec-4ceb-a383-216177534bb7} TCPHandler: Code: 159. DB::Exception: Distributed DDL task /test/test_t1fje8px/rdb/log/query-0000000009 is not finished on 3 of 4 hosts (0 of them are currently executing the task, 2 are inactive). They are going to execute the query in background. Was waiting for 5.675952418 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::DDLQueryStatusSource::generate() @ 0x0000000029b11829 4. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 5. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 10. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d3463e7 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:42.503903 [ 10342 ] {b47c5b85-bfb4-4ab2-85ea-6b0a051f751a} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:34418) (comment: 00180_no_seek_avoiding_when_reading_from_cache.sh) (in query: SELECT `__table1`.`ProfileEvents`['AsynchronousReaderIgnoredBytes'] AS `arrayElement(ProfileEvents, 'AsynchronousReaderIgnoredBytes')`, `__table1`.`ProfileEvents`['CachedReadBufferReadFromSourceBytes'] AS `arrayElement(ProfileEvents, 'CachedReadBufferReadFromSourceBytes')` FROM `system`.`query_log` AS `__table1` WHERE (`__table1`.`query_id` = '02906_read_from_cache_26614') AND (`__table1`.`type` = 'QueryFinish') AND (`__table1`.`event_date` >= _CAST(20355, 'Date')) AND (`__table1`.`current_database` = _CAST('test_9', 'String'))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:42.509220 [ 10342 ] {b47c5b85-bfb4-4ab2-85ea-6b0a051f751a} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:42.595856 [ 2994 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:42.666765 [ 2997 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:42.703848 [ 8396 ] {2775608e-c776-4995-9fa4-2028fd9b8e58} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=0): (while reading column current_database): (while reading from part /var/lib/clickhouse/store/f81/f81dbc1c-a1bd-4968-9514-f0c97137169a/202509_110_110_0/ in table system.query_log (f81dbc1c-a1bd-4968-9514-f0c97137169a) located on disk default of type local, from mark 0 with max_rows_to_read = 88): While executing MergeTreeSelect(pool: ReadPoolParallelReplicas, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:42826) (comment: 00180_no_seek_avoiding_when_reading_from_cache.sh) (in query: SELECT `__table1`.`ProfileEvents`['AsynchronousReaderIgnoredBytes'] AS `arrayElement(ProfileEvents, 'AsynchronousReaderIgnoredBytes')`, `__table1`.`ProfileEvents`['CachedReadBufferReadFromSourceBytes'] AS `arrayElement(ProfileEvents, 'CachedReadBufferReadFromSourceBytes')` FROM `system`.`query_log` AS `__table1` WHERE (`__table1`.`query_id` = '02906_read_from_cache_26614') AND (`__table1`.`type` = 'QueryFinish') AND (`__table1`.`event_date` >= _CAST(20355, 'Date')) AND (`__table1`.`current_database` = _CAST('test_9', 'String'))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. ./src/IO/ReadBuffer.h:0: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x000000002775f11b 13. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:22: DB::CompressedReadBufferFromFile::nextImpl() @ 0x0000000027763d41 14. DB::ReadBuffer::next() @ 0x000000000b528aeb 15. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x000000000b581216 16. ./contrib/llvm-project/libcxx/include/new:246: DB::SerializationLowCardinality::deserializeBinaryBulkStatePrefix(DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>*) const @ 0x0000000027b57db7 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::readPrefix(DB::NameAndTypePair const&, std::shared_ptr const&, std::shared_ptr&, std::function const&) @ 0x000000002c7b4491 18. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c7b3d56 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.cpp:0: DB::MergeTreeReaderCompactSingleBuffer::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7eb73e 20. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 21. ./src/Storages/MergeTree/MergeTreeRangeReader.h:139: DB::MergeTreeRangeReader::continueReadingChain(DB::MergeTreeRangeReader::ReadResult const&, unsigned long&) @ 0x000000002c7df38d 22. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:984: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbd9d 23. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbd35 24. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 25. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:38: DB::MergeTreeThreadSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002de66094 26. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 27. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 28. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 29. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 30. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 31. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 2025.09.25 21:21:42.723768 [ 8396 ] {2775608e-c776-4995-9fa4-2028fd9b8e58} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=0): (while reading column current_database): (while reading from part /var/lib/clickhouse/store/f81/f81dbc1c-a1bd-4968-9514-f0c97137169a/202509_110_110_0/ in table system.query_log (f81dbc1c-a1bd-4968-9514-f0c97137169a) located on disk default of type local, from mark 0 with max_rows_to_read = 88): While executing MergeTreeSelect(pool: ReadPoolParallelReplicas, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. ./src/IO/ReadBuffer.h:0: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x000000002775f11b 13. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:22: DB::CompressedReadBufferFromFile::nextImpl() @ 0x0000000027763d41 14. DB::ReadBuffer::next() @ 0x000000000b528aeb 15. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x000000000b581216 16. ./contrib/llvm-project/libcxx/include/new:246: DB::SerializationLowCardinality::deserializeBinaryBulkStatePrefix(DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>*) const @ 0x0000000027b57db7 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::readPrefix(DB::NameAndTypePair const&, std::shared_ptr const&, std::shared_ptr&, std::function const&) @ 0x000000002c7b4491 18. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c7b3d56 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.cpp:0: DB::MergeTreeReaderCompactSingleBuffer::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7eb73e 20. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 21. ./src/Storages/MergeTree/MergeTreeRangeReader.h:139: DB::MergeTreeRangeReader::continueReadingChain(DB::MergeTreeRangeReader::ReadResult const&, unsigned long&) @ 0x000000002c7df38d 22. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:984: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbd9d 23. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbd35 24. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 25. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:38: DB::MergeTreeThreadSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002de66094 26. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 27. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 28. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 29. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 30. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 31. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 2025.09.25 21:21:42.742893 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:42.754542 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:42.760020 [ 13129 ] {6b1a0290-7ef8-4136-a116-69045702a0e1} executeQuery: Code: 439. DB::Exception: Received from 127.0.0.1:9000. DB::Exception: Cannot schedule a task: failed to start the thread (threads=2, jobs=2). Stack trace: 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 : While executing Remote. (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:35928) (comment: 00180_no_seek_avoiding_when_reading_from_cache.sh) (in query: -- AsynchronousReaderIgnoredBytes = 0: no seek-avoiding happened -- CachedReadBufferReadFromSourceBytes = 0: sanity check to ensure we read only from cache SELECT ProfileEvents['AsynchronousReaderIgnoredBytes'], ProfileEvents['CachedReadBufferReadFromSourceBytes'] FROM system.query_log WHERE query_id = '02906_read_from_cache_26614' AND type = 'QueryFinish' AND event_date >= yesterday() AND current_database = currentDatabase() ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001aeb974e 3. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000002cee6c6a 4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x000000002cef780e 5. ./build_docker/./src/Client/MultiplexedConnections.cpp:340: DB::MultiplexedConnections::receivePacketUnlocked(std::function) @ 0x000000002cf8e09e 6. ./build_docker/./src/QueryPipeline/RemoteQueryExecutorReadContext.cpp:0: DB::RemoteQueryExecutorReadContext::Task::run(std::function, std::function) @ 0x00000000278ad616 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278ab436 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278aaa44 9. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func>>(unsigned int, unsigned int) @ 0x00000000278a8394 2025.09.25 21:21:42.768931 [ 8396 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::ffff:127.0.0.1]:42826, local: [::ffff:127.0.0.2]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:42.770766 [ 10537 ] {} ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket (peer: [::ffff:127.0.0.1]:47500, local: [::ffff:127.0.0.7]:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:83: DB::ReadBufferFromPocoSocketBase::socketReceiveBytesImpl(char*, unsigned long) @ 0x000000001b0b6042 4. ./build_docker/./src/IO/ReadBufferFromPocoSocket.cpp:107: DB::ReadBufferFromPocoSocketBase::nextImpl() @ 0x000000001b0b6b84 5. ./build_docker/./src/IO/ReadBufferFromPocoSocketChunked.cpp:103: DB::ReadBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf045f6 6. DB::ReadBuffer::next() @ 0x000000000b528aeb 7. ./build_docker/./src/Server/TCPHandler.cpp:383: DB::TCPHandler::runImpl() @ 0x000000002d174fd3 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:42.773433 [ 13129 ] {6b1a0290-7ef8-4136-a116-69045702a0e1} TCPHandler: Code: 439. DB::Exception: Received from 127.0.0.1:9000. DB::Exception: Cannot schedule a task: failed to start the thread (threads=2, jobs=2). Stack trace: 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 : While executing Remote. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001aeb974e 3. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000002cee6c6a 4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x000000002cef780e 5. ./build_docker/./src/Client/MultiplexedConnections.cpp:340: DB::MultiplexedConnections::receivePacketUnlocked(std::function) @ 0x000000002cf8e09e 6. ./build_docker/./src/QueryPipeline/RemoteQueryExecutorReadContext.cpp:0: DB::RemoteQueryExecutorReadContext::Task::run(std::function, std::function) @ 0x00000000278ad616 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278ab436 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278aaa44 9. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func>>(unsigned int, unsigned int) @ 0x00000000278a8394 2025.09.25 21:21:42.931079 [ 5082 ] {} DDLWorker(rdb_test_t1fje8px_3): Error on initialization of rdb_test_t1fje8px_3: Code: 57. DB::Exception: Directory for table data store/8e5/8e521274-511c-4053-a286-6b6551a2f792/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:42.987896 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:43.380811 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:43.394371 [ 3013 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:43.597574 [ 3020 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:43.668544 [ 3025 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:43.697048 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:43.736623 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:43.835540 [ 13130 ] {22dcb459-ba2c-4e5f-8112-3479cf9b56ca} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:35872) (comment: 02404_schema_inference_cache_respect_format_settings.gen.sql) (in query: insert into function file(02404_data.CustomSeparated) select number, toDate(number) from numbers(10);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d3463e7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:43.840555 [ 13130 ] {22dcb459-ba2c-4e5f-8112-3479cf9b56ca} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d3463e7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:44.396660 [ 3041 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:44.461677 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:44.600007 [ 3052 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:44.656682 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:44.671191 [ 3053 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:44.692148 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:45.338092 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:45.390506 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:45.399503 [ 3080 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:45.556460 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:45.615007 [ 3073 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:45.676820 [ 3089 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:46.032549 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:46.032089 [ 10342 ] {81c19270-50a6-459f-8650-1f6df9ca7b6e} executeQuery: Code: 57. DB::Exception: Cannot attach table with UUID 3182ece8-f087-4253-b9a4-6ce2ef4e1ef4, because it was detached but still used by some query. Retry later. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:36096) (comment: 01107_atomic_db_detach_attach.sh) (in query: ATTACH TABLE test_01107_test_3.mt), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception, DB::UUIDTag> const&>(int, FormatStringHelperImpl, DB::UUIDTag> const&>::type>, StrongTypedef, DB::UUIDTag> const&) @ 0x0000000027d02954 4. ./build_docker/./src/Databases/DatabaseAtomic.cpp:399: DB::DatabaseAtomic::assertDetachedTableNotInUse(StrongTypedef, DB::UUIDTag> const&) @ 0x0000000027cf2dd4 5. ./build_docker/./src/Databases/DatabaseAtomic.cpp:0: DB::DatabaseAtomic::checkDetachedTableNotInUse(StrongTypedef, DB::UUIDTag> const&) @ 0x0000000027d0163a 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291b8727 7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:46.039453 [ 10342 ] {81c19270-50a6-459f-8650-1f6df9ca7b6e} TCPHandler: Code: 57. DB::Exception: Cannot attach table with UUID 3182ece8-f087-4253-b9a4-6ce2ef4e1ef4, because it was detached but still used by some query. Retry later. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception, DB::UUIDTag> const&>(int, FormatStringHelperImpl, DB::UUIDTag> const&>::type>, StrongTypedef, DB::UUIDTag> const&) @ 0x0000000027d02954 4. ./build_docker/./src/Databases/DatabaseAtomic.cpp:399: DB::DatabaseAtomic::assertDetachedTableNotInUse(StrongTypedef, DB::UUIDTag> const&) @ 0x0000000027cf2dd4 5. ./build_docker/./src/Databases/DatabaseAtomic.cpp:0: DB::DatabaseAtomic::checkDetachedTableNotInUse(StrongTypedef, DB::UUIDTag> const&) @ 0x0000000027d0163a 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291b8727 7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 8. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:46.054833 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:46.347071 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:46.401939 [ 3105 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:46.609774 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:46.618550 [ 3103 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:46.655339 [ 12045 ] {6d610d0c-d8e1-4e76-adc4-0739852301ec} executeQuery: Code: 219. DB::Exception: Database test_01107_test_3 cannot be detached, because some tables are still in use. Retry later. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:36350) (comment: 01107_atomic_db_detach_attach.sh) (in query: DETACH DATABASE test_01107_test_3), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Databases/DatabaseAtomic.cpp:441: DB::DatabaseAtomic::assertCanBeDetached(bool) @ 0x0000000027cfb400 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:393: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5572 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:46.661577 [ 12045 ] {6d610d0c-d8e1-4e76-adc4-0739852301ec} TCPHandler: Code: 219. DB::Exception: Database test_01107_test_3 cannot be detached, because some tables are still in use. Retry later. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Databases/DatabaseAtomic.cpp:441: DB::DatabaseAtomic::assertCanBeDetached(bool) @ 0x0000000027cfb400 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:393: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5572 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:46.679772 [ 3098 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:47.104150 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:47.121572 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:47.171075 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:47.308823 [ 5440 ] {} DDLWorker(rdb_test_t1fje8px_2): Error on initialization of rdb_test_t1fje8px_2: Code: 57. DB::Exception: Directory for table data store/8e5/8e521274-511c-4053-a286-6b6551a2f792/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:47.404724 [ 3125 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:47.427112 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:47.460690 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:47.621017 [ 3150 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:47.687163 [ 3154 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:47.847535 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:48.027887 [ 13130 ] {cdd34eff-1149-44fc-b323-4c013e4c9e8e} executeQuery: Code: 48. DB::Exception: Truncate is not supported by storage Values. (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from 0.0.0.0:0) (comment: 02188_table_function_format.sql) (in query: /* ddl_entry=query-0000000076 */ TRUNCATE TABLE test_5.test), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageProxy::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000265c80a2 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 13. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 14. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 15. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f029c 16. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 17. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 18. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 19. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 20. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 22. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 26. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 27. asan_thread_start(void*) @ 0x000000000aa43059 28. ? @ 0x00007f12cd75eac3 29. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:48.149189 [ 13130 ] {cdd34eff-1149-44fc-b323-4c013e4c9e8e} executeQuery: Code: 48. DB::Exception: Truncate is not supported by storage Values. (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:36098) (comment: 02188_table_function_format.sql) (in query: drop table test;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageProxy::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000265c80a2 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 13. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 14. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 15. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f029c 16. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 17. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 18. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 19. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 20. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 22. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 26. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 27. asan_thread_start(void*) @ 0x000000000aa43059 28. ? @ 0x00007f12cd75eac3 29. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:48.156430 [ 13130 ] {cdd34eff-1149-44fc-b323-4c013e4c9e8e} TCPHandler: Code: 48. DB::Exception: Truncate is not supported by storage Values. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageProxy::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000265c80a2 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 13. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 14. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 15. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f029c 16. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 17. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 18. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 19. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 20. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 22. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 26. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 27. asan_thread_start(void*) @ 0x000000000aa43059 28. ? @ 0x00007f12cd75eac3 29. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:48.255863 [ 5082 ] {} DDLWorker(rdb_test_t1fje8px_3): Error on initialization of rdb_test_t1fje8px_3: Code: 57. DB::Exception: Directory for table data store/8e5/8e521274-511c-4053-a286-6b6551a2f792/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:48.339899 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:48.379167 [ 13129 ] {7b0e4ea9-a277-4c18-a78c-e5e256b21d01} executeQuery: Code: 159. DB::Exception: Distributed DDL task /test/test_t1fje8px/rdb/log/query-0000000010 is not finished on 3 of 4 hosts (0 of them are currently executing the task, 2 are inactive). They are going to execute the query in background. Was waiting for 5.039957756 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:35986) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_t1fje8px.t33 (n int) engine=Log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::DDLQueryStatusSource::generate() @ 0x0000000029b11829 4. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 5. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 10. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:48.382388 [ 13129 ] {7b0e4ea9-a277-4c18-a78c-e5e256b21d01} TCPHandler: Code: 159. DB::Exception: Distributed DDL task /test/test_t1fje8px/rdb/log/query-0000000010 is not finished on 3 of 4 hosts (0 of them are currently executing the task, 2 are inactive). They are going to execute the query in background. Was waiting for 5.039957756 seconds, which is longer than distributed_ddl_task_timeout. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::DDLQueryStatusSource::generate() @ 0x0000000029b11829 4. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 5. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 6. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 10. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:48.407567 [ 3172 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:48.505495 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:48.546096 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:48.623864 [ 3182 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:48.625058 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:48.690816 [ 3183 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:48.865587 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:49.163940 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:49.410756 [ 3207 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:49.626586 [ 3203 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:49.696572 [ 3205 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:49.848210 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:49.896149 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:50.105776 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:50.348561 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:50.413872 [ 3225 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:50.629967 [ 3227 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:50.700251 [ 3232 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:50.873344 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:51.063146 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:51.103070 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:51.417041 [ 3258 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:51.496231 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:51.532978 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:51.632404 [ 3259 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:51.703373 [ 2760 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:51.752865 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:52.174824 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:52.348214 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:52.412065 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:52.421833 [ 2780 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:52.507406 [ 5440 ] {} DDLWorker(rdb_test_t1fje8px_2): Error on initialization of rdb_test_t1fje8px_2: Code: 57. DB::Exception: Directory for table data store/be9/be91eab8-506f-46b6-8326-5da4ae667eea/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:52.635552 [ 2751 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:52.707710 [ 2786 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:53.103786 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:53.423175 [ 5082 ] {} DDLWorker(rdb_test_t1fje8px_3): Error on initialization of rdb_test_t1fje8px_3: Code: 57. DB::Exception: Directory for table data store/8e5/8e521274-511c-4053-a286-6b6551a2f792/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:53.424240 [ 2798 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:53.569311 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:53.582072 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:53.638559 [ 2796 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:53.709800 [ 2813 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:53.719565 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:54.129619 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:54.148977 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:54.212832 [ 10342 ] {01600_log_queries_with_extensive_info_004_err} executeQuery: Code: 32. DB::Exception: Attempt to read after eof: while converting '' to Int32. (ATTEMPT_TO_READ_AFTER_EOF) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:36428) (comment: 01600_log_queries_with_extensive_info.sh) (in query: select k from test_log_queries.logtable where i > ''), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/IO/VarInt.cpp:13: DB::throwReadAfterEOF() @ 0x000000001aefe9cb 5. ./src/IO/ReadHelpers.h:530: void DB::readIntTextUnsafe(long&, DB::ReadBuffer&) @ 0x0000000027ba3a7c 6. ./build_docker/./src/DataTypes/Serializations/SerializationNumber.cpp:0: DB::SerializationNumber::deserializeText(DB::IColumn&, DB::ReadBuffer&, DB::FormatSettings const&, bool) const @ 0x0000000027ba35d3 7. ./build_docker/./src/Interpreters/convertFieldToType.cpp:554: DB::(anonymous namespace)::convertFieldToTypeImpl(DB::Field const&, DB::IDataType const&, DB::IDataType const*) @ 0x0000000029aa606f 8. ./build_docker/./src/Interpreters/convertFieldToType.cpp:611: DB::convertFieldToType(DB::Field const&, DB::IDataType const&, DB::IDataType const*) @ 0x0000000029aa2141 9. DB::FunctionComparison::executeWithConstString(std::shared_ptr const&, DB::IColumn const*, DB::IColumn const*, std::shared_ptr const&, std::shared_ptr const&, unsigned long) const @ 0x00000000100bdf10 10. DB::FunctionComparison::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000001008bfd1 11. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 12. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 13. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 14. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 15. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 16. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::ActionsDAG::evaluatePartialResult(std::unordered_map, std::equal_to, std::allocator>>&, std::vector> const&, unsigned long, bool) @ 0x0000000028259428 17. ./contrib/llvm-project/libcxx/include/vector:961: DB::ActionsDAG::updateHeader(DB::Block const&) const @ 0x0000000028256e17 18. ./build_docker/./src/Processors/Transforms/FilterTransform.cpp:0: DB::FilterTransform::transformHeader(DB::Block const&, DB::ActionsDAG const*, String const&, bool) @ 0x000000002d9c9921 19. ./build_docker/./src/Processors/QueryPlan/FilterStep.cpp:0: DB::FilterStep::FilterStep(DB::DataStream const&, DB::ActionsDAG, String, bool) @ 0x000000002ddb09e1 20. ./contrib/llvm-project/libcxx/include/string:1499: std::__unique_if::__unique_single std::make_unique[abi:v15007](DB::DataStream const&, DB::ActionsDAG&&, String&, bool&) @ 0x00000000293c12e7 21. ./build_docker/./src/Planner/Planner.cpp:0: DB::(anonymous namespace)::addFilterStep(DB::QueryPlan&, DB::FilterAnalysisResult&, String const&, std::unordered_set, std::hash>, std::equal_to>, std::allocator>>&) @ 0x000000002943b3f5 22. ./contrib/llvm-project/libcxx/include/string:1499: DB::Planner::buildPlanForQueryNode() @ 0x000000002942e6e9 23. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 24. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 28. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 2025.09.25 21:21:54.215792 [ 10342 ] {01600_log_queries_with_extensive_info_004_err} TCPHandler: Code: 32. DB::Exception: Attempt to read after eof: while converting '' to Int32. (ATTEMPT_TO_READ_AFTER_EOF), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/IO/VarInt.cpp:13: DB::throwReadAfterEOF() @ 0x000000001aefe9cb 5. ./src/IO/ReadHelpers.h:530: void DB::readIntTextUnsafe(long&, DB::ReadBuffer&) @ 0x0000000027ba3a7c 6. ./build_docker/./src/DataTypes/Serializations/SerializationNumber.cpp:0: DB::SerializationNumber::deserializeText(DB::IColumn&, DB::ReadBuffer&, DB::FormatSettings const&, bool) const @ 0x0000000027ba35d3 7. ./build_docker/./src/Interpreters/convertFieldToType.cpp:554: DB::(anonymous namespace)::convertFieldToTypeImpl(DB::Field const&, DB::IDataType const&, DB::IDataType const*) @ 0x0000000029aa606f 8. ./build_docker/./src/Interpreters/convertFieldToType.cpp:611: DB::convertFieldToType(DB::Field const&, DB::IDataType const&, DB::IDataType const*) @ 0x0000000029aa2141 9. DB::FunctionComparison::executeWithConstString(std::shared_ptr const&, DB::IColumn const*, DB::IColumn const*, std::shared_ptr const&, std::shared_ptr const&, unsigned long) const @ 0x00000000100bdf10 10. DB::FunctionComparison::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000001008bfd1 11. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 12. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 13. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 14. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 15. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 16. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::ActionsDAG::evaluatePartialResult(std::unordered_map, std::equal_to, std::allocator>>&, std::vector> const&, unsigned long, bool) @ 0x0000000028259428 17. ./contrib/llvm-project/libcxx/include/vector:961: DB::ActionsDAG::updateHeader(DB::Block const&) const @ 0x0000000028256e17 18. ./build_docker/./src/Processors/Transforms/FilterTransform.cpp:0: DB::FilterTransform::transformHeader(DB::Block const&, DB::ActionsDAG const*, String const&, bool) @ 0x000000002d9c9921 19. ./build_docker/./src/Processors/QueryPlan/FilterStep.cpp:0: DB::FilterStep::FilterStep(DB::DataStream const&, DB::ActionsDAG, String, bool) @ 0x000000002ddb09e1 20. ./contrib/llvm-project/libcxx/include/string:1499: std::__unique_if::__unique_single std::make_unique[abi:v15007](DB::DataStream const&, DB::ActionsDAG&&, String&, bool&) @ 0x00000000293c12e7 21. ./build_docker/./src/Planner/Planner.cpp:0: DB::(anonymous namespace)::addFilterStep(DB::QueryPlan&, DB::FilterAnalysisResult&, String const&, std::unordered_set, std::hash>, std::equal_to>, std::allocator>>&) @ 0x000000002943b3f5 22. ./contrib/llvm-project/libcxx/include/string:1499: DB::Planner::buildPlanForQueryNode() @ 0x000000002942e6e9 23. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 24. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 28. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 2025.09.25 21:21:54.479916 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:54.497697 [ 2825 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:54.530211 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:54.641827 [ 2834 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:54.723931 [ 2840 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:54.854427 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:55.083357 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:55.288043 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:55.305705 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:55.442344 [ 8172 ] {} DDLWorker(rdb_test_t1fje8px_9): Error on initialization of rdb_test_t1fje8px_9: Code: 57. DB::Exception: Directory for table data store/be9/be91eab8-506f-46b6-8326-5da4ae667eea/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:55.501285 [ 2859 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:55.570631 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:55.595877 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:55.644842 [ 2867 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:55.726691 [ 2873 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:55.777945 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:56.057752 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:56.504011 [ 2888 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:56.648226 [ 2891 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:56.661149 [ 10538 ] {01600_log_queries_with_extensive_info_007} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:36500) (comment: 01600_log_queries_with_extensive_info.sh) (in query: create table test_log_queries.logtable2 as test_log_queries.logtable), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c45d4f6 8. ./contrib/llvm-project/libcxx/include/optional:260: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>) @ 0x000000002cbacf28 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1460: std::shared_ptr std::allocate_shared[abi:v15007], DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>, void>(std::allocator const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>&&) @ 0x000000002cbabafa 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::create(DB::StorageFactory::Arguments const&) @ 0x000000002cba21b8 11. ./build_docker/./src/Storages/StorageFactory.cpp:225: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x000000002b77bf53 12. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1659: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291b9254 13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:56.675922 [ 10538 ] {01600_log_queries_with_extensive_info_007} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c45d4f6 8. ./contrib/llvm-project/libcxx/include/optional:260: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>) @ 0x000000002cbacf28 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1460: std::shared_ptr std::allocate_shared[abi:v15007], DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>, void>(std::allocator const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>&&) @ 0x000000002cbabafa 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::create(DB::StorageFactory::Arguments const&) @ 0x000000002cba21b8 11. ./build_docker/./src/Storages/StorageFactory.cpp:225: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x000000002b77bf53 12. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1659: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291b9254 13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:21:56.729670 [ 2886 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:56.911431 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:56.947422 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:57.035898 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:57.507259 [ 2914 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:57.654068 [ 2911 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:57.701256 [ 5440 ] {} DDLWorker(rdb_test_t1fje8px_2): Error on initialization of rdb_test_t1fje8px_2: Code: 57. DB::Exception: Directory for table data store/8e5/8e521274-511c-4053-a286-6b6551a2f792/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:57.737155 [ 2923 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:57.896420 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:58.070039 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:58.152370 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:58.512775 [ 2947 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:58.656793 [ 2954 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:58.666402 [ 5082 ] {} DDLWorker(rdb_test_t1fje8px_3): Error on initialization of rdb_test_t1fje8px_3: Code: 57. DB::Exception: Directory for table data store/8e5/8e521274-511c-4053-a286-6b6551a2f792/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:58.772616 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:58.802723 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:58.820578 [ 2948 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:59.006201 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:59.108826 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:59.138443 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:59.221933 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:59.519816 [ 2968 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:59.658962 [ 2979 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:59.824030 [ 2987 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:21:59.995749 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:00.222986 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:00.261329 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:00.271247 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:00.530102 [ 3001 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:00.661607 [ 2999 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:00.763219 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:00.780181 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:00.827170 [ 3011 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:01.209306 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:01.298984 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:01.398340 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:01.533957 [ 3032 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:01.664856 [ 3036 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:01.802583 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:01.824671 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:01.829360 [ 3043 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:01.981694 [ 10540 ] {664ee7bd-8c98-4508-b9ea-809d27615f59} executeQuery: Code: 57. DB::Exception: Table test_5.test_empty_order_by already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from 0.0.0.0:0) (comment: 02904_empty_order_by_with_setting_enabled.sh) (in query: /* ddl_entry=query-0000000079 */ CREATE TABLE test_5.test_empty_order_by UUID 'd031bd12-d494-49c7-a7ae-993efa32f26a' (`a` UInt8, `b` String) ENGINE = MergeTree PRIMARY KEY (a, b) SETTINGS index_granularity = 8192), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:02.027455 [ 10540 ] {664ee7bd-8c98-4508-b9ea-809d27615f59} executeQuery: Code: 57. DB::Exception: Table test_5.test_empty_order_by already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:41140) (comment: 02904_empty_order_by_with_setting_enabled.sh) (in query: CREATE TABLE test_empty_order_by(a UInt8 PRIMARY KEY, b String PRIMARY KEY) ENGINE = MergeTree() SETTINGS index_granularity = 8192;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:02.035548 [ 10540 ] {664ee7bd-8c98-4508-b9ea-809d27615f59} TCPHandler: Code: 57. DB::Exception: Table test_5.test_empty_order_by already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:02.130763 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:02.181055 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:02.228900 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:02.540113 [ 3060 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:02.546583 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:02.592722 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:02.668465 [ 3062 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:02.804267 [ 5440 ] {} DDLWorker(rdb_test_t1fje8px_2): Error on initialization of rdb_test_t1fje8px_2: Code: 57. DB::Exception: Directory for table data store/8e5/8e521274-511c-4053-a286-6b6551a2f792/ already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1600: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc318 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Databases/DatabaseReplicated.cpp:1368: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&)::$_4::operator()() const @ 0x0000000027d74704 8. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::DatabaseReplicated::recoverLostReplica(std::shared_ptr const&, unsigned int, unsigned int&) @ 0x0000000027d69c13 9. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000027da20dd 10. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9de6b 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 12. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:02.832297 [ 3075 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:02.889521 [ 10931 ] {cd4c37ef-55a9-4a0b-9049-1587a13b7058} executeQuery: Code: 57. DB::Exception: Table test_5.test_empty_order_by already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from 0.0.0.0:0) (comment: 02904_empty_order_by_with_setting_enabled.sh) (in query: /* ddl_entry=query-0000000080 */ CREATE TABLE test_5.test_empty_order_by UUID '5bac28a9-6e4b-48e8-b753-ffc7d19fd857' (`a` UInt8, `b` String) ENGINE = MergeTree PRIMARY KEY a SETTINGS index_granularity = 8192), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:03.014393 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:03.033301 [ 10931 ] {cd4c37ef-55a9-4a0b-9049-1587a13b7058} executeQuery: Code: 57. DB::Exception: Table test_5.test_empty_order_by already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:41162) (comment: 02904_empty_order_by_with_setting_enabled.sh) (in query: CREATE TABLE test_empty_order_by(a UInt8, b String) ENGINE = MergeTree() PRIMARY KEY (a) SETTINGS index_granularity = 8192;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:03.048028 [ 10931 ] {cd4c37ef-55a9-4a0b-9049-1587a13b7058} TCPHandler: Code: 57. DB::Exception: Table test_5.test_empty_order_by already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:03.325782 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:03.398388 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:03.542548 [ 3082 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:03.699316 [ 3102 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:03.842876 [ 3105 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:04.020122 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:04.068271 [ 10540 ] {26740e7e-8c06-43b0-ba6f-93c22a88c85e} executeQuery: Code: 57. DB::Exception: Table test_5.test_empty_order_by already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from 0.0.0.0:0) (comment: 02904_empty_order_by_with_setting_enabled.sh) (in query: /* ddl_entry=query-0000000081 */ CREATE TABLE test_5.test_empty_order_by UUID '8a71da4f-99ee-4744-8217-f182f721c468' (`a` UInt8, `b` String) ENGINE = MergeTree ORDER BY (a, b) SETTINGS index_granularity = 8192), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:04.103650 [ 10540 ] {26740e7e-8c06-43b0-ba6f-93c22a88c85e} executeQuery: Code: 57. DB::Exception: Table test_5.test_empty_order_by already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:41196) (comment: 02904_empty_order_by_with_setting_enabled.sh) (in query: CREATE TABLE test_empty_order_by(a UInt8, b String) ENGINE = MergeTree() ORDER BY (a, b) SETTINGS index_granularity = 8192;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:04.117673 [ 10540 ] {26740e7e-8c06-43b0-ba6f-93c22a88c85e} TCPHandler: Code: 57. DB::Exception: Table test_5.test_empty_order_by already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:04.515950 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:04.546683 [ 3124 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:04.577024 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:04.710396 [ 3129 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:04.846196 [ 3127 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:04.988104 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:05.276159 [ 10540 ] {f5ac5db1-27df-46f2-bec5-49f2b2f08935} DatabaseReplicated (rdb_test_t1fje8px_3): Database rdb_test_t1fje8px_3 exists locally, but its replica does not exist in ZooKeeper (/test/test_t1fje8px/rdb/replicas/s2|r1). Assuming it was dropped, will not try to start it up 2025.09.25 21:22:05.296490 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:05.319351 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:05.548767 [ 3140 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:05.713568 [ 3156 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:05.849110 [ 3157 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:05.941489 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:06.010675 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:06.113835 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:06.169273 [ 13100 ] {0beec22d-1fc6-477b-9a4b-f56bae8bcbff} executeQuery: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:41242) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_t1fje8px_3.t2 as system.query_log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:06.187451 [ 13100 ] {0beec22d-1fc6-477b-9a4b-f56bae8bcbff} TCPHandler: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:06.405335 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:06.541219 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:06.551873 [ 3189 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:06.554065 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:06.717184 [ 3196 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:06.791682 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:06.879220 [ 3199 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:07.445247 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:07.509450 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:07.555318 [ 3217 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:07.601141 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:07.720400 [ 3219 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:07.929754 [ 3220 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:08.136821 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:08.220026 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:08.583025 [ 3243 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:08.681251 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:08.723467 [ 3234 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:08.843058 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:08.932721 [ 3245 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:08.956269 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:09.586702 [ 2749 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:09.603060 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:09.726600 [ 2754 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:09.856537 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:09.935276 [ 2763 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:10.043175 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:10.108877 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:10.242498 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:10.504136 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:10.589532 [ 2792 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:10.676945 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:10.734454 [ 2784 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:10.755842 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:10.900144 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:10.938313 [ 2791 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:11.301336 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:11.514179 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:11.592758 [ 2820 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:11.736878 [ 2811 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:11.847558 [ 10546 ] {e4671e4f-4916-4eca-a1e4-3a9c54e42cfb} executeQuery: Code: 241. DB::Exception: Memory tracker (for query): fault injected. Would use 7.76 MiB (attempt to allocate chunk of 0 bytes), maximum: 9.31 GiB: While executing AggregatingTransform. (MEMORY_LIMIT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:41176) (comment: 00172_hits_joins.gen.sql) (in query: SELECT domain, hits, visits FROM ( SELECT domain(URL) AS domain, count() AS hits FROM test.hits GROUP BY domain ) ANY LEFT JOIN ( SELECT domain(StartURL) AS domain, sum(Sign) AS visits FROM test.visits GROUP BY domain ) USING domain ORDER BY hits DESC LIMIT 10 SETTINGS joined_subquery_requires_alias = 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, char const*&&, char const*&&, String&&, long&, String&&) @ 0x000000001adfc3f5 4. ./build_docker/./src/Common/MemoryTracker.cpp:284: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfafab 5. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfa777 6. ./build_docker/./src/Interpreters/Aggregator.cpp:0: DB::Aggregator::checkLimits(unsigned long, bool&) const @ 0x00000000287bd6f2 7. ./build_docker/./src/Interpreters/Aggregator.cpp:0: DB::Aggregator::executeOnBlock(std::vector::immutable_ptr, std::allocator::immutable_ptr>>, unsigned long, unsigned long, DB::AggregatedDataVariants&, std::vector>&, std::vector>, std::allocator>>>&, bool&) const @ 0x00000000287d0c7c 8. ./build_docker/./src/Processors/Transforms/AggregatingTransform.cpp:692: DB::AggregatingTransform::consume(DB::Chunk) @ 0x000000002d964575 9. ./build_docker/./src/Processors/Transforms/AggregatingTransform.cpp:0: DB::AggregatingTransform::work() @ 0x000000002d95e675 10. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 14. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 15. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:11.857177 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:11.865764 [ 10546 ] {e4671e4f-4916-4eca-a1e4-3a9c54e42cfb} TCPHandler: Code: 241. DB::Exception: Memory tracker (for query): fault injected. Would use 7.76 MiB (attempt to allocate chunk of 0 bytes), maximum: 9.31 GiB: While executing AggregatingTransform. (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, char const*&&, char const*&&, String&&, long&, String&&) @ 0x000000001adfc3f5 4. ./build_docker/./src/Common/MemoryTracker.cpp:284: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfafab 5. ./build_docker/./src/Common/MemoryTracker.cpp:0: MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001adfa777 6. ./build_docker/./src/Interpreters/Aggregator.cpp:0: DB::Aggregator::checkLimits(unsigned long, bool&) const @ 0x00000000287bd6f2 7. ./build_docker/./src/Interpreters/Aggregator.cpp:0: DB::Aggregator::executeOnBlock(std::vector::immutable_ptr, std::allocator::immutable_ptr>>, unsigned long, unsigned long, DB::AggregatedDataVariants&, std::vector>&, std::vector>, std::allocator>>>&, bool&) const @ 0x00000000287d0c7c 8. ./build_docker/./src/Processors/Transforms/AggregatingTransform.cpp:692: DB::AggregatingTransform::consume(DB::Chunk) @ 0x000000002d964575 9. ./build_docker/./src/Processors/Transforms/AggregatingTransform.cpp:0: DB::AggregatingTransform::work() @ 0x000000002d95e675 10. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 14. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 15. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:11.902420 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:11.943338 [ 2822 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:12.524135 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:12.596597 [ 2835 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:12.678329 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:12.683157 [ 8396 ] {90c05b95-bad5-4cc6-94c6-253d5e828b20} DatabaseReplicated (rdb_test_t1fje8px_2): Database rdb_test_t1fje8px_2 exists locally, but its replica does not exist in ZooKeeper (/test/test_t1fje8px/rdb/replicas/s1|r2). Assuming it was dropped, will not try to start it up 2025.09.25 21:22:12.739816 [ 2824 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:12.743043 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:12.946932 [ 2856 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:13.034785 [ 11796 ] {9a5582ce-65d4-4507-98ab-f9359f6a4f32} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): (while reading column data.out.:`Array(JSON)`): (while reading from part /var/lib/clickhouse/store/f2a/f2ab393d-f82e-43b0-a88c-1dcdad145078/all_1_1_0/ in table test_5.t_json_6 (f2ab393d-f82e-43b0-a88c-1dcdad145078) located on disk default of type local, from mark 0 with max_rows_to_read = 2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:49938) (comment: 01825_new_type_json_6.sh) (in query: SELECT DISTINCT arrayJoin(JSONAllPathsWithTypes(arrayJoin(data.out[]))) as path FROM t_json_6 order by path;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:312: DB::MergeTreeReaderWide::getStream(bool, DB::ISerialization::SubstreamPath const&, DB::MergeTreeDataPartChecksums const&, DB::NameAndTypePair const&, unsigned long, bool, unsigned long, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&) @ 0x000000002c803f22 11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:401: DB::ReadBuffer* std::__function::__policy_invoker::__call_impl const&, COW::immutable_ptr&, unsigned long, bool, unsigned long, unsigned long, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>&, bool)::$_0, DB::ReadBuffer* (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c8056e5 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027ab9944 13. ./build_docker/./src/DataTypes/Serializations/SerializationVariantElement.cpp:0: DB::SerializationVariantElement::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027c6922c 14. ./build_docker/./src/DataTypes/Serializations/SerializationDynamicElement.cpp:0: DB::SerializationDynamicElement::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027b15a85 15. ./build_docker/./src/DataTypes/Serializations/SerializationObjectDynamicPath.cpp:0: DB::SerializationObjectDynamicPath::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027c0065a 16. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:196: DB::MergeTreeReaderWide::readData(DB::NameAndTypePair const&, std::shared_ptr const&, COW::immutable_ptr&, unsigned long, bool, unsigned long, unsigned long, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>&, bool) @ 0x000000002c80211e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:158: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c800e8d 18. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 19. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000002c7e0473 20. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbec3 21. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 22. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:53: DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002dea5374 23. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 24. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 25. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 26. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 27. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 28. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 30. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 31. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 2025.09.25 21:22:13.069702 [ 11796 ] {9a5582ce-65d4-4507-98ab-f9359f6a4f32} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): (while reading column data.out.:`Array(JSON)`): (while reading from part /var/lib/clickhouse/store/f2a/f2ab393d-f82e-43b0-a88c-1dcdad145078/all_1_1_0/ in table test_5.t_json_6 (f2ab393d-f82e-43b0-a88c-1dcdad145078) located on disk default of type local, from mark 0 with max_rows_to_read = 2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:312: DB::MergeTreeReaderWide::getStream(bool, DB::ISerialization::SubstreamPath const&, DB::MergeTreeDataPartChecksums const&, DB::NameAndTypePair const&, unsigned long, bool, unsigned long, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&) @ 0x000000002c803f22 11. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:401: DB::ReadBuffer* std::__function::__policy_invoker::__call_impl const&, COW::immutable_ptr&, unsigned long, bool, unsigned long, unsigned long, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>&, bool)::$_0, DB::ReadBuffer* (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c8056e5 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027ab9944 13. ./build_docker/./src/DataTypes/Serializations/SerializationVariantElement.cpp:0: DB::SerializationVariantElement::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027c6922c 14. ./build_docker/./src/DataTypes/Serializations/SerializationDynamicElement.cpp:0: DB::SerializationDynamicElement::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027b15a85 15. ./build_docker/./src/DataTypes/Serializations/SerializationObjectDynamicPath.cpp:0: DB::SerializationObjectDynamicPath::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027c0065a 16. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:196: DB::MergeTreeReaderWide::readData(DB::NameAndTypePair const&, std::shared_ptr const&, COW::immutable_ptr&, unsigned long, bool, unsigned long, unsigned long, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>&, bool) @ 0x000000002c80211e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:158: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c800e8d 18. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 19. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000002c7e0473 20. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbec3 21. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 22. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:53: DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002dea5374 23. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 24. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 25. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 26. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 27. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 28. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 30. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 31. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 2025.09.25 21:22:13.476302 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:13.599940 [ 2865 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:13.652672 [ 11796 ] {9367f133-16e0-4d99-89ee-62e254d68784} executeQuery: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:49954) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_t1fje8px_2.t2 as system.query_log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:13.671517 [ 11796 ] {9367f133-16e0-4d99-89ee-62e254d68784} TCPHandler: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:13.695226 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:13.725586 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:13.741784 [ 2872 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:13.949206 [ 2870 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:14.231135 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:14.284599 [ 11796 ] {04085074-6869-40ac-bad6-2bce034034ae} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:49968) (comment: 03130_analyzer_array_join_prefer_column.sql) (in query: CREATE TABLE test_table (`id` UInt64, `value` String, `value_array` Array(UInt64)) ENGINE = MergeTree ORDER BY id SETTINGS index_granularity = 33193, min_bytes_for_wide_part = 1073741824, ratio_of_defaults_for_sparse_serialization = 0.07894133031368256, replace_long_file_name_to_hash = true, max_file_name_length = 0, min_bytes_for_full_part_storage = 0, compact_parts_max_bytes_to_buffer = 68535099, compact_parts_max_granules_to_buffer = 167, compact_parts_merge_max_bytes_to_prefetch_part = 1188221, merge_max_block_size = 14228, old_parts_lifetime = 10., prefer_fetch_merged_part_size_threshold = 10737418240, vertical_merge_algorithm_min_rows_to_activate = 420931, vertical_merge_algorithm_min_columns_to_activate = 100, min_merge_bytes_to_use_direct_io = 1, index_granularity_bytes = 8420213, concurrent_part_removal_threshold = 0, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = true, marks_compress_block_size = 40556, primary_key_compress_block_size = 31308), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c45d4f6 8. ./contrib/llvm-project/libcxx/include/optional:260: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>) @ 0x000000002cbacf28 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1460: std::shared_ptr std::allocate_shared[abi:v15007], DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>, void>(std::allocator const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>&&) @ 0x000000002cbabafa 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::create(DB::StorageFactory::Arguments const&) @ 0x000000002cba21b8 11. ./build_docker/./src/Storages/StorageFactory.cpp:225: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x000000002b77bf53 12. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1659: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291b9254 13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:14.297654 [ 11796 ] {04085074-6869-40ac-bad6-2bce034034ae} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c45d4f6 8. ./contrib/llvm-project/libcxx/include/optional:260: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>) @ 0x000000002cbacf28 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1460: std::shared_ptr std::allocate_shared[abi:v15007], DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>, void>(std::allocator const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>&&) @ 0x000000002cbabafa 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962: DB::create(DB::StorageFactory::Arguments const&) @ 0x000000002cba21b8 11. ./build_docker/./src/Storages/StorageFactory.cpp:225: DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x000000002b77bf53 12. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1659: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291b9254 13. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:14.476823 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:14.603098 [ 2896 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:14.746924 [ 2901 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:14.908730 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:14.945390 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:14.951884 [ 2895 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:15.462414 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:15.678567 [ 2914 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:15.708828 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:15.750096 [ 2922 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:15.955077 [ 2923 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:15.977766 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:16.005086 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:16.192943 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:16.681716 [ 2942 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:16.752899 [ 2947 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:16.957700 [ 2954 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:17.053226 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:17.068799 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:17.096124 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:17.635216 [ 10931 ] {1318fb46-ae32-4b53-b03c-1f809fec528c} executeQuery: Code: 36. DB::Exception: Element of set in IN, VALUES, or LIMIT, or aggregate function parameter, or a table function argument is not a constant expression (result column not found): arrayJoin([0, 1]). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60256) (comment: 02009_array_join_partition.sql) (in query: ALTER TABLE table_2009_part ATTACH PARTITION tuple(arrayJoin([0, 1]));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x000000000aafc574 4. ./build_docker/./src/Interpreters/evaluateConstantExpression.cpp:105: DB::evaluateConstantExpressionImpl(std::shared_ptr const&, std::shared_ptr const&, bool) @ 0x0000000029adb886 5. ./contrib/llvm-project/libcxx/include/optional:344: DB::evaluateConstantExpression(std::shared_ptr const&, std::shared_ptr const&) @ 0x0000000029adc621 6. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::getPartitionIDFromQuery(std::shared_ptr const&, std::shared_ptr, DB::DataPartsLock*) const @ 0x000000002c49f848 7. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:5030: DB::MergeTreeData::checkAlterPartitionIsPossible(std::vector> const&, std::shared_ptr const&, DB::Settings const&, std::shared_ptr) const @ 0x000000002c4cf3f1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000029174ac0 9. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 13. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:17.641930 [ 10931 ] {1318fb46-ae32-4b53-b03c-1f809fec528c} TCPHandler: Code: 36. DB::Exception: Element of set in IN, VALUES, or LIMIT, or aggregate function parameter, or a table function argument is not a constant expression (result column not found): arrayJoin([0, 1]). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x000000000aafc574 4. ./build_docker/./src/Interpreters/evaluateConstantExpression.cpp:105: DB::evaluateConstantExpressionImpl(std::shared_ptr const&, std::shared_ptr const&, bool) @ 0x0000000029adb886 5. ./contrib/llvm-project/libcxx/include/optional:344: DB::evaluateConstantExpression(std::shared_ptr const&, std::shared_ptr const&) @ 0x0000000029adc621 6. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::getPartitionIDFromQuery(std::shared_ptr const&, std::shared_ptr, DB::DataPartsLock*) const @ 0x000000002c49f848 7. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:5030: DB::MergeTreeData::checkAlterPartitionIsPossible(std::vector> const&, std::shared_ptr const&, DB::Settings const&, std::shared_ptr) const @ 0x000000002c4cf3f1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000029174ac0 9. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 13. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:17.688550 [ 2960 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:17.759217 [ 2967 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:17.827939 [ 10931 ] {b6aff2a5-5901-44f2-8f5e-a016a52587e4} executeQuery: Code: 36. DB::Exception: Element of set in IN, VALUES, or LIMIT, or aggregate function parameter, or a table function argument is not a constant expression (result column not found): toYYYYMM(toDate(array(arrayJoin(array(arrayJoin(array(arrayJoin(array(arrayJoin(array(3, materialize(NULL), arrayJoin(array(1025, materialize(NULL), materialize(NULL))), NULL)))))), materialize(NULL)))), NULL)). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60256) (comment: 02009_array_join_partition.sql) (in query: ALTER TABLE table_2009_part ATTACH PARTITION tuple(toYYYYMM(toDate([arrayJoin([arrayJoin([arrayJoin([arrayJoin([3, materialize(NULL), arrayJoin([1025, materialize(NULL), materialize(NULL)]), NULL])])]), materialize(NULL)])], NULL)));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x000000000aafc574 4. ./build_docker/./src/Interpreters/evaluateConstantExpression.cpp:105: DB::evaluateConstantExpressionImpl(std::shared_ptr const&, std::shared_ptr const&, bool) @ 0x0000000029adb886 5. ./contrib/llvm-project/libcxx/include/optional:344: DB::evaluateConstantExpression(std::shared_ptr const&, std::shared_ptr const&) @ 0x0000000029adc621 6. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::getPartitionIDFromQuery(std::shared_ptr const&, std::shared_ptr, DB::DataPartsLock*) const @ 0x000000002c49f848 7. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:5030: DB::MergeTreeData::checkAlterPartitionIsPossible(std::vector> const&, std::shared_ptr const&, DB::Settings const&, std::shared_ptr) const @ 0x000000002c4cf3f1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000029174ac0 9. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 13. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:17.928082 [ 10931 ] {b6aff2a5-5901-44f2-8f5e-a016a52587e4} TCPHandler: Code: 36. DB::Exception: Element of set in IN, VALUES, or LIMIT, or aggregate function parameter, or a table function argument is not a constant expression (result column not found): toYYYYMM(toDate(array(arrayJoin(array(arrayJoin(array(arrayJoin(array(arrayJoin(array(3, materialize(NULL), arrayJoin(array(1025, materialize(NULL), materialize(NULL))), NULL)))))), materialize(NULL)))), NULL)). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x000000000aafc574 4. ./build_docker/./src/Interpreters/evaluateConstantExpression.cpp:105: DB::evaluateConstantExpressionImpl(std::shared_ptr const&, std::shared_ptr const&, bool) @ 0x0000000029adb886 5. ./contrib/llvm-project/libcxx/include/optional:344: DB::evaluateConstantExpression(std::shared_ptr const&, std::shared_ptr const&) @ 0x0000000029adc621 6. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::getPartitionIDFromQuery(std::shared_ptr const&, std::shared_ptr, DB::DataPartsLock*) const @ 0x000000002c49f848 7. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:5030: DB::MergeTreeData::checkAlterPartitionIsPossible(std::vector> const&, std::shared_ptr const&, DB::Settings const&, std::shared_ptr) const @ 0x000000002c4cf3f1 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000029174ac0 9. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 13. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:17.952463 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:17.983902 [ 2979 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:18.062101 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:18.158083 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:18.408010 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:18.518072 [ 10931 ] {8cd82859-4381-4e59-90ae-83f4e18e51d6} DatabaseReplicated (rdb_test_t1fje8px): Database rdb_test_t1fje8px exists locally, but its replica does not exist in ZooKeeper (/test/test_t1fje8px/rdb/replicas/s1|r1). Assuming it was dropped, will not try to start it up 2025.09.25 21:22:18.691015 [ 2995 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:18.763567 [ 2933 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:18.965252 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:18.986752 [ 3004 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:19.116748 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:19.142871 [ 11796 ] {a7e4aa29-c67b-4e65-8af2-fe3a48249231} executeQuery: Code: 291. DB::Exception: Path 'data_*_{_partition_id}.csv' contains globs, so the table is in readonly mode. (DATABASE_ACCESS_DENIED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60294) (comment: 03037_s3_write_to_globbed_partitioned_path.sql) (in query: insert into function s3('http://localhost:11111/test/data_*_{_partition_id}.csv') partition by number % 3 select * from numbers(10);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:310: DB::StorageObjectStorage::write(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, bool) @ 0x00000000267b9cbd 5. ./build_docker/./src/Processors/Transforms/buildPushingToViewsChain.cpp:573: DB::buildPushingToViewsChain(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr const&, bool, std::shared_ptr, std::shared_ptr, std::atomic*, bool, DB::Block const&) @ 0x000000002db6fb52 6. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:342: DB::InterpreterInsertQuery::buildSink(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::atomic*) @ 0x000000002935079f 7. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:427: DB::InterpreterInsertQuery::buildPreAndSinkChains(unsigned long, unsigned long, std::shared_ptr, std::shared_ptr const&, DB::Block const&) @ 0x0000000029352541 8. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:625: DB::InterpreterInsertQuery::buildInsertSelectPipeline(DB::ASTInsertQuery&, std::shared_ptr) @ 0x0000000029355dd4 9. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:789: DB::InterpreterInsertQuery::execute() @ 0x000000002935b5a0 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 13. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:19.174550 [ 11796 ] {a7e4aa29-c67b-4e65-8af2-fe3a48249231} TCPHandler: Code: 291. DB::Exception: Path 'data_*_{_partition_id}.csv' contains globs, so the table is in readonly mode. (DATABASE_ACCESS_DENIED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:310: DB::StorageObjectStorage::write(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, bool) @ 0x00000000267b9cbd 5. ./build_docker/./src/Processors/Transforms/buildPushingToViewsChain.cpp:573: DB::buildPushingToViewsChain(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr const&, bool, std::shared_ptr, std::shared_ptr, std::atomic*, bool, DB::Block const&) @ 0x000000002db6fb52 6. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:342: DB::InterpreterInsertQuery::buildSink(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::atomic*) @ 0x000000002935079f 7. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:427: DB::InterpreterInsertQuery::buildPreAndSinkChains(unsigned long, unsigned long, std::shared_ptr, std::shared_ptr const&, DB::Block const&) @ 0x0000000029352541 8. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:625: DB::InterpreterInsertQuery::buildInsertSelectPipeline(DB::ASTInsertQuery&, std::shared_ptr) @ 0x0000000029355dd4 9. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:789: DB::InterpreterInsertQuery::execute() @ 0x000000002935b5a0 10. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 13. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:19.459268 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:19.465766 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:19.505083 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:19.693564 [ 3023 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:19.766002 [ 3035 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:19.882819 [ 11796 ] {2442acff-a102-480b-91e8-a726043fb379} executeQuery: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60310) (comment: 02447_drop_database_replica.sh) (in query: create table rdb_test_t1fje8px.t2 as system.query_log), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:19.901377 [ 11796 ] {2442acff-a102-480b-91e8-a726043fb379} TCPHandler: Code: 225. DB::Exception: Database is in readonly mode, because it cannot connect to ZooKeeper. (NO_ZOOKEEPER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:993: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d60cf8 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:19.989680 [ 3038 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:20.260722 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:20.531664 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:20.594338 [ 5073 ] {60e7f98b-e52a-47f3-a4c0-137a20ab48ac} void DB::ParallelParsingInputFormat::onBackgroundException(): Code: 117. DB::Exception: Size of BSON document is invalid. (INCORRECT_DATA), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Processors/Formats/Impl/BSONEachRowRowInputFormat.cpp:1041: DB::fileSegmentationEngineBSONEachRow(DB::ReadBuffer&, DB::Memory>&, unsigned long, unsigned long) @ 0x000000002d49e91e 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002d6f5fdc 6. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000002d6ff2d2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 9. asan_thread_start(void*) @ 0x000000000aa43059 10. ? @ 0x00007f12cd75eac3 11. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:20.625938 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:20.629233 [ 8396 ] {60e7f98b-e52a-47f3-a4c0-137a20ab48ac} executeQuery: Code: 117. DB::Exception: Size of BSON document is invalid: While executing ParallelParsingBlockInputFormat. (INCORRECT_DATA) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60314) (comment: 02589_bson_invalid_document_size.sql) (in query: select * from format(BSONEachRow, 'x UInt32', x'00000000');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Processors/Formats/Impl/BSONEachRowRowInputFormat.cpp:1041: DB::fileSegmentationEngineBSONEachRow(DB::ReadBuffer&, DB::Memory>&, unsigned long, unsigned long) @ 0x000000002d49e91e 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002d6f5fdc 6. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000002d6ff2d2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 9. asan_thread_start(void*) @ 0x000000000aa43059 10. ? @ 0x00007f12cd75eac3 11. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:20.632633 [ 8396 ] {60e7f98b-e52a-47f3-a4c0-137a20ab48ac} TCPHandler: Code: 117. DB::Exception: Size of BSON document is invalid: While executing ParallelParsingBlockInputFormat. (INCORRECT_DATA), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Processors/Formats/Impl/BSONEachRowRowInputFormat.cpp:1041: DB::fileSegmentationEngineBSONEachRow(DB::ReadBuffer&, DB::Memory>&, unsigned long, unsigned long) @ 0x000000002d49e91e 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002d6f5fdc 6. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000002d6ff2d2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 9. asan_thread_start(void*) @ 0x000000000aa43059 10. ? @ 0x00007f12cd75eac3 11. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:20.696677 [ 3059 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:20.769156 [ 3058 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:20.897595 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:20.993069 [ 3057 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:21.295474 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:21.333728 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:21.405436 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:21.701454 [ 3081 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:21.772766 [ 3090 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:21.888026 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:21.964727 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:21.996289 [ 3100 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:22.002763 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:22.368720 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:22.440344 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:22.704112 [ 3127 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:22.724315 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:22.758764 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:22.777408 [ 3141 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:22.779193 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:23.000693 [ 3136 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:23.337993 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:23.741246 [ 3161 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:23.780081 [ 3165 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:23.938382 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:23.993436 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:24.004417 [ 3175 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:24.372361 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:24.394362 [ 13110 ] {} TCPHandler: Code: 452. DB::Exception: Setting allow_experimental_object_type should not be changed. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/remove_if.h:32: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f80932 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./build_docker/./src/Server/TCPHandler.cpp:2034: DB::TCPHandler::receiveQuery() @ 0x000000002d1a17ad 10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::receivePacket() @ 0x000000002d1877e2 11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d175061 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:24.681092 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:24.739088 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:24.745589 [ 3199 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:24.839229 [ 3207 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:25.009011 [ 3169 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:25.029860 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:25.357816 [ 13129 ] {fbfa5c37-e657-4d1f-893b-11e2ac53b6dd} executeQuery: Code: 60. DB::Exception: Table test_3.t_json_7 does not exist. (UNKNOWN_TABLE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60380) (comment: 01825_type_json_7.sh) (in query: INSERT INTO t_json_7 FORMAT JSONAsObject), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr) const @ 0x0000000027dd1ade 5. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr, std::optional*) const @ 0x00000000286af00b 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:801: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr) const @ 0x00000000286bb117 7. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:0: DB::InterpreterInsertQuery::getTable(DB::ASTInsertQuery&) @ 0x000000002934b836 8. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:748: DB::InterpreterInsertQuery::execute() @ 0x000000002935ad49 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:25.377306 [ 13129 ] {fbfa5c37-e657-4d1f-893b-11e2ac53b6dd} TCPHandler: Code: 60. DB::Exception: Table test_3.t_json_7 does not exist. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr) const @ 0x0000000027dd1ade 5. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr, std::optional*) const @ 0x00000000286af00b 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:801: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr) const @ 0x00000000286bb117 7. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:0: DB::InterpreterInsertQuery::getTable(DB::ASTInsertQuery&) @ 0x000000002934b836 8. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:748: DB::InterpreterInsertQuery::execute() @ 0x000000002935ad49 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:25.731130 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:25.747997 [ 3223 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:25.841815 [ 3226 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:25.949166 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:25.980804 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:26.012287 [ 3218 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:26.074577 [ 8396 ] {52a2b54c-9de6-44a6-aab9-ae0e49be5249} executeQuery: Code: 60. DB::Exception: Unknown table expression identifier 't_json_7' in scope SELECT DISTINCT toTypeName(data) FROM t_json_7. (UNKNOWN_TABLE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60400) (comment: 01825_type_json_7.sh) (in query: SELECT DISTINCT toTypeName(data) FROM t_json_7;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:26.089816 [ 8396 ] {52a2b54c-9de6-44a6-aab9-ae0e49be5249} TCPHandler: Code: 60. DB::Exception: Unknown table expression identifier 't_json_7' in scope SELECT DISTINCT toTypeName(data) FROM t_json_7. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:26.395440 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:26.746667 [ 8396 ] {ee085027-92a2-4e7b-940f-a96fa9ac077f} executeQuery: Code: 60. DB::Exception: Unknown table expression identifier 't_json_7' in scope SELECT data.key, data.categories FROM t_json_7 ORDER BY data.key ASC. (UNKNOWN_TABLE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37694) (comment: 01825_type_json_7.sh) (in query: SELECT data.key, data.categories FROM t_json_7 ORDER BY data.key), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:26.749768 [ 8396 ] {ee085027-92a2-4e7b-940f-a96fa9ac077f} TCPHandler: Code: 60. DB::Exception: Unknown table expression identifier 't_json_7' in scope SELECT data.key, data.categories FROM t_json_7 ORDER BY data.key ASC. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:26.754530 [ 3256 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:26.821669 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:26.843825 [ 3245 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:27.017607 [ 3255 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:27.031899 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:27.065304 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:27.411358 [ 8396 ] {3a13dc24-d765-4e46-a0c8-74ce0ebf140c} executeQuery: Code: 60. DB::Exception: Table test_3.t_json_7 does not exist. (UNKNOWN_TABLE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37718) (comment: 01825_type_json_7.sh) (in query: DROP TABLE t_json_7;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr) const @ 0x0000000027dd1ade 5. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr, std::optional*) const @ 0x00000000286af00b 6. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:969: DB::DatabaseCatalog::getDatabaseAndTable(DB::StorageID const&, std::shared_ptr) const @ 0x00000000286bb62c 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291eeb31 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 14. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 18. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:27.415579 [ 8396 ] {3a13dc24-d765-4e46-a0c8-74ce0ebf140c} TCPHandler: Code: 60. DB::Exception: Table test_3.t_json_7 does not exist. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr) const @ 0x0000000027dd1ade 5. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr, std::optional*) const @ 0x00000000286af00b 6. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:969: DB::DatabaseCatalog::getDatabaseAndTable(DB::StorageID const&, std::shared_ptr) const @ 0x00000000286bb62c 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291eeb31 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 14. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 18. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:27.481134 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:27.527304 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:27.625837 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:27.758324 [ 2751 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:27.850575 [ 2788 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:27.937284 [ 3362 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {f81dbc1c-a1bd-4968-9514-f0c97137169a::202509_1_117_23}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:27.941955 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:28.019778 [ 2794 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:28.099979 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:28.533015 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:28.563290 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:28.760742 [ 2809 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:28.844595 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:28.853541 [ 2811 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:29.036539 [ 2825 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:29.306216 [ 10543 ] {ea5dbfc1-d1d1-4a46-ae7a-cafd5cbb0a4f} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:45518) (comment: 00177_memory_bound_merging.sh) (in query: SELECT count() > 0 AS `greater(COUNT(), 0)` FROM `system`.`text_log` AS `__table1` WHERE (`__table1`.`query_id` IN (SELECT `__table1`.`query_id` AS `query_id` FROM `system`.`query_log` AS `__table1` WHERE (`__table1`.`query_id` != 'query_id_memory_bound_merging_2043910945') AND (`__table1`.`initial_query_id` = 'query_id_memory_bound_merging_2043910945') AND (`__table1`.`event_date` >= _CAST(20355, 'Date')))) AND (`__table1`.`event_date` >= _CAST(20355, 'Date')) AND (`__table1`.`message` ILIKE '%Reading%ranges in order%')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:29.310447 [ 10543 ] {ea5dbfc1-d1d1-4a46-ae7a-cafd5cbb0a4f} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:29.334912 [ 13129 ] {fc76797f-204d-4f53-8bc2-41af38f95e87} executeQuery: Code: 439. DB::Exception: Received from 127.0.0.5:9000. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). Stack trace: 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 : While executing Remote. (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37734) (comment: 00177_memory_bound_merging.sh) (in query: SELECT COUNT() > 0 FROM system.text_log WHERE query_id IN (SELECT query_id FROM system.query_log WHERE query_id != 'query_id_memory_bound_merging_2043910945' AND initial_query_id = 'query_id_memory_bound_merging_2043910945' AND event_date >= yesterday()) AND event_date >= yesterday() AND message ILIKE '%Reading%ranges in order%'), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001aeb974e 3. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000002cee6c6a 4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x000000002cef780e 5. ./build_docker/./src/Client/MultiplexedConnections.cpp:340: DB::MultiplexedConnections::receivePacketUnlocked(std::function) @ 0x000000002cf8e09e 6. ./build_docker/./src/QueryPipeline/RemoteQueryExecutorReadContext.cpp:0: DB::RemoteQueryExecutorReadContext::Task::run(std::function, std::function) @ 0x00000000278ad616 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278ab436 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278aaa44 9. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func>>(unsigned int, unsigned int) @ 0x00000000278a8394 2025.09.25 21:22:29.493088 [ 13129 ] {fc76797f-204d-4f53-8bc2-41af38f95e87} TCPHandler: Code: 439. DB::Exception: Received from 127.0.0.5:9000. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). Stack trace: 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 : While executing Remote. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001aeb974e 3. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000002cee6c6a 4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x000000002cef780e 5. ./build_docker/./src/Client/MultiplexedConnections.cpp:340: DB::MultiplexedConnections::receivePacketUnlocked(std::function) @ 0x000000002cf8e09e 6. ./build_docker/./src/QueryPipeline/RemoteQueryExecutorReadContext.cpp:0: DB::RemoteQueryExecutorReadContext::Task::run(std::function, std::function) @ 0x00000000278ad616 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278ab436 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278aaa44 9. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func>>(unsigned int, unsigned int) @ 0x00000000278a8394 2025.09.25 21:22:29.573321 [ 3682 ] {9515f4fa-da4f-45a0-bea3-fdb508e62b03} executeQuery: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:41142). (NETWORK_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:41142) (comment: 00177_memory_bound_merging.sh) (in query: SELECT count() > 0 AS `greater(COUNT(), 0)` FROM `system`.`text_log` AS `__table1` WHERE (`__table1`.`query_id` IN (SELECT `__table1`.`query_id` AS `query_id` FROM `system`.`query_log` AS `__table1` WHERE (`__table1`.`query_id` != 'query_id_memory_bound_merging_2043910945') AND (`__table1`.`initial_query_id` = 'query_id_memory_bound_merging_2043910945') AND (`__table1`.`event_date` >= _CAST(20355, 'Date')))) AND (`__table1`.`event_date` >= _CAST(20355, 'Date')) AND (`__table1`.`message` ILIKE '%Reading%ranges in order%')), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001b0bada2 4. ./build_docker/./src/IO/WriteBufferFromPocoSocketChunked.cpp:201: DB::WriteBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf09bf7 5. ./src/IO/WriteBuffer.h:66: DB::TCPHandler::sendProgress() @ 0x000000002d18c7b2 6. ./build_docker/./src/Server/TCPHandler.cpp:1170: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18be00 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:29.579023 [ 3682 ] {9515f4fa-da4f-45a0-bea3-fdb508e62b03} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:41142). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001b0bada2 4. ./build_docker/./src/IO/WriteBufferFromPocoSocketChunked.cpp:201: DB::WriteBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf09bf7 5. ./src/IO/WriteBuffer.h:66: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000002d1a9beb 6. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x000000002d18db59 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d176902 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:29.581530 [ 3682 ] {9515f4fa-da4f-45a0-bea3-fdb508e62b03} TCPHandler: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::ffff:127.0.0.10]:9000 -> [::ffff:127.0.0.1]:41142). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001b0bada2 4. ./build_docker/./src/IO/WriteBufferFromPocoSocketChunked.cpp:201: DB::WriteBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf09bf7 5. ./src/IO/WriteBuffer.h:66: DB::TCPHandler::sendProgress() @ 0x000000002d18c7b2 6. ./build_docker/./src/Server/TCPHandler.cpp:1170: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18be00 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:29.591675 [ 3682 ] {9515f4fa-da4f-45a0-bea3-fdb508e62b03} TCPHandler: Client has gone away. 2025.09.25 21:22:29.728211 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:29.756674 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:29.768516 [ 2848 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:29.856362 [ 2852 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:29.991722 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:30.040657 [ 2864 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:30.494793 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:30.760307 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:30.770372 [ 2877 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:30.808984 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:30.858803 [ 2874 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:30.923324 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:31.044631 [ 2889 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:31.251443 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:31.282434 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:31.772118 [ 2899 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:31.860992 [ 2912 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:31.863448 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:32.047003 [ 2921 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:32.157559 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:32.195029 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:32.580463 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:32.615195 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:32.717093 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:32.776441 [ 2940 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:32.863875 [ 2944 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:33.049871 [ 2936 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:33.432386 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:33.461851 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:33.606414 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:33.779132 [ 2955 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:33.817384 [ 10537 ] {7bc55d87-7282-47b7-825c-98b812ec1667} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1): While executing ConvertingAggregatedToChunksTransform. (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37766) (comment: 00344_row_number_in_all_blocks.sql) (in query: SELECT groupUniqArray(blockSize()), uniqExact(rowNumberInAllBlocks()), min(rowNumberInAllBlocks()), max(rowNumberInAllBlocks()), uniqExact(rowNumberInBlock()), min(rowNumberInBlock()), max(rowNumberInBlock()), uniqExact(blockNumber()) FROM (SELECT * FROM system.numbers_mt LIMIT 100000);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000024a55a45 7. ./contrib/llvm-project/libcxx/include/vector:590: DB::AggregateFunctionUniq>::parallelizeMergePrepare(std::vector>&, ThreadPoolImpl>&, std::atomic&) const @ 0x0000000024a4c0d5 8. ./contrib/llvm-project/libcxx/include/vector:590: DB::Aggregator::mergeWithoutKeyDataImpl(std::vector, std::allocator>>&, std::atomic&) const @ 0x00000000287e3a36 9. ./build_docker/./src/Processors/Transforms/AggregatingTransform.cpp:467: DB::ConvertingAggregatedToChunksTransform::initialize() @ 0x000000002d96e51e 10. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 14. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 15. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:33.866580 [ 2961 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:33.897550 [ 10537 ] {7bc55d87-7282-47b7-825c-98b812ec1667} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1): While executing ConvertingAggregatedToChunksTransform. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000024a55a45 7. ./contrib/llvm-project/libcxx/include/vector:590: DB::AggregateFunctionUniq>::parallelizeMergePrepare(std::vector>&, ThreadPoolImpl>&, std::atomic&) const @ 0x0000000024a4c0d5 8. ./contrib/llvm-project/libcxx/include/vector:590: DB::Aggregator::mergeWithoutKeyDataImpl(std::vector, std::allocator>>&, std::atomic&) const @ 0x00000000287e3a36 9. ./build_docker/./src/Processors/Transforms/AggregatingTransform.cpp:467: DB::ConvertingAggregatedToChunksTransform::initialize() @ 0x000000002d96e51e 10. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 11. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 12. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 13. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 14. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 15. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:33.953183 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:34.052086 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:34.053102 [ 2969 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:34.163205 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:34.466286 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:34.517478 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:34.561160 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:34.785437 [ 2988 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:34.866916 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:34.869562 [ 2991 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:34.892682 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:35.022330 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:35.060943 [ 2998 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:35.720169 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:35.747383 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:35.779062 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:35.788369 [ 3027 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:35.872956 [ 3021 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:35.984655 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:36.064359 [ 3037 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:36.098406 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:36.109754 [ 3354 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {02260661-95d0-4eca-a73d-2543d99ce5b9::202509_189_269_15}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:36.231428 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:36.697580 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:36.791573 [ 3054 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:36.876273 [ 3066 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:36.883535 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:36.919984 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:37.011529 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:37.066648 [ 3071 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:37.338160 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:37.358170 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:37.505227 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:37.589477 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:37.605217 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:37.793778 [ 3103 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:37.880113 [ 3111 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:38.069916 [ 3113 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:38.356550 [ 8396 ] {2bc53b27-9a56-4cc2-9a13-e50c18a0ff35} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42118) (comment: 00169_contingency.sql) (in query: WITH ClientIP AS a, RemoteIP AS b SELECT round(cramersV(a, b), 2), round(cramersVBiasCorrected(a, b), 2), round(theilsU(a, b), 2), round(theilsU(b, a), 2), round(contingency(a, b), 2) FROM test.hits;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:38.368460 [ 8396 ] {2bc53b27-9a56-4cc2-9a13-e50c18a0ff35} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:38.469252 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:38.531209 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:38.555686 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:38.795482 [ 3128 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:38.890661 [ 3130 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:39.073030 [ 3151 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:39.114424 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:39.505830 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:39.556884 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:39.593144 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:39.798147 [ 3143 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:39.864468 [ 13112 ] {7a7363e2-4882-4e66-9f55-b13a0f715c6b} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42140) (comment: 00042_any_left_join.sql) (in query: SELECT EventDate, hits, visits FROM ( SELECT EventDate, count() AS hits FROM test.hits GROUP BY EventDate ) ANY LEFT JOIN ( SELECT StartDate AS EventDate, sum(Sign) AS visits FROM test.visits GROUP BY EventDate ) USING EventDate ORDER BY hits DESC LIMIT 10 SETTINGS joined_subquery_requires_alias = 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:39.881699 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:39.893376 [ 3166 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:40.035185 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:40.035670 [ 13112 ] {7a7363e2-4882-4e66-9f55-b13a0f715c6b} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=9, jobs=9). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:40.076022 [ 3165 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:40.109015 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:40.801476 [ 3189 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:40.896627 [ 3196 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:40.973765 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:41.137160 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:41.078087 [ 3188 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:41.167251 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:41.804488 [ 3193 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:41.865360 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:41.952548 [ 3214 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:42.044600 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:42.060024 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:42.148791 [ 3223 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:42.283501 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:42.558153 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:42.570248 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:42.807836 [ 3231 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:42.955815 [ 3244 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:43.078655 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:43.094981 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:43.098508 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:43.151924 [ 3248 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:43.242155 [ 8396 ] {8ae89b1a-1300-4a53-ab14-a8cf40037857} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=732). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42146) (comment: 00424_shard_aggregate_functions_of_nullable.sql) (in query: SELECT quantileExactWeighted(0.5)(x, y) FROM ( SELECT CAST(1 AS Nullable(UInt8)) AS x, CAST(0 AS Nullable(UInt8)) AS y UNION ALL SELECT CAST(NULL AS Nullable(UInt8)) AS x, CAST(1 AS Nullable(UInt8)) AS y UNION ALL SELECT CAST(2 AS Nullable(UInt8)) AS x, CAST(NULL AS Nullable(UInt8)) AS y UNION ALL SELECT CAST(number AS Nullable(UInt8)) AS x, CAST(number AS Nullable(UInt8)) AS y FROM system.numbers LIMIT 10 );), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:43.250095 [ 8396 ] {8ae89b1a-1300-4a53-ab14-a8cf40037857} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=732). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:43.585001 [ 13129 ] {990a1a0b-173c-40df-829d-a26823eb55b9} executeQuery: Code: 1. DB::Exception: JOIN , ... right table expression expected to process query to fetch columns stage. Actual WithMergeableState. (UNSUPPORTED_METHOD) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42176) (comment: 00181_cross_join_compression.sql) (in query: SELECT CounterID, StartURL FROM unit, test.visits ORDER BY (CounterID, StartURL) DESC LIMIT 1000;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, char const*&&) @ 0x000000001b512ffa 4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1219: DB::(anonymous namespace)::buildQueryPlanForJoinNode(std::shared_ptr const&, DB::JoinTreeQueryPlan, DB::JoinTreeQueryPlan, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x0000000029479365 5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::buildJoinTreeQueryPlan(std::shared_ptr const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002945f174 6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000002942c986 7. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:43.593048 [ 13129 ] {990a1a0b-173c-40df-829d-a26823eb55b9} TCPHandler: Code: 1. DB::Exception: JOIN , ... right table expression expected to process query to fetch columns stage. Actual WithMergeableState. (UNSUPPORTED_METHOD), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, char const*&&) @ 0x000000001b512ffa 4. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1219: DB::(anonymous namespace)::buildQueryPlanForJoinNode(std::shared_ptr const&, DB::JoinTreeQueryPlan, DB::JoinTreeQueryPlan, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x0000000029479365 5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:0: DB::buildJoinTreeQueryPlan(std::shared_ptr const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set, std::equal_to, std::allocator> const&, std::shared_ptr&) @ 0x000000002945f174 6. ./build_docker/./src/Planner/Planner.cpp:0: DB::Planner::buildPlanForQueryNode() @ 0x000000002942c986 7. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 8. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:43.641455 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:43.717621 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:43.773557 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:43.813469 [ 2756 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:43.942266 [ 10537 ] {6cd06d33-655b-4758-b240-9727a47e60cc} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42162) (comment: 00148_monotonic_functions_and_index.sql) (in query: SELECT count() FROM test.hits WHERE CounterID <= 732797;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:43.951056 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:43.953972 [ 10537 ] {6cd06d33-655b-4758-b240-9727a47e60cc} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:43.958887 [ 2762 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:44.154806 [ 2767 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:44.391330 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:44.492911 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:44.591340 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:44.818624 [ 2778 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:44.940419 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:44.956030 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:44.961743 [ 2758 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:45.158424 [ 2794 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:45.658167 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:45.821489 [ 2806 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:45.860259 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:45.997808 [ 2815 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:46.168930 [ 2827 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:46.190574 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:46.235800 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:46.733284 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:46.823690 [ 2846 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:47.001977 [ 2854 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:47.065298 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:47.097341 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:47.171742 [ 2858 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:47.601998 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:47.636263 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:47.763879 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:47.825854 [ 2871 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:48.008245 [ 2879 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:48.174229 [ 2890 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:48.269231 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:48.649860 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:48.828708 [ 2902 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:48.952596 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:48.997100 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:49.012075 [ 2908 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:49.038824 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:49.177246 [ 2900 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:49.832817 [ 2920 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:49.848967 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:49.889737 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:50.014455 [ 2923 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:50.163281 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:50.179354 [ 2929 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:50.254140 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:50.390293 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:50.594248 [ 10537 ] {b454aa44-6c2f-4d62-9247-d603b97ef9b0} executeQuery: Code: 48. DB::Exception: Truncate is not supported by storage View. (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:45974) (comment: 00056_view.sql) (in query: DROP TABLE test.view;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:50.607566 [ 10537 ] {b454aa44-6c2f-4d62-9247-d603b97ef9b0} TCPHandler: Code: 48. DB::Exception: Truncate is not supported by storage View. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:50.650254 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:50.686290 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:50.834894 [ 2948 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:50.974568 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:51.016991 [ 2956 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:51.181201 [ 2965 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:51.304921 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:51.702444 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:51.742393 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:51.837095 [ 2975 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:52.019999 [ 2994 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:52.075302 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:52.183145 [ 2995 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:52.208166 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:52.231877 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:52.840096 [ 3011 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:52.938678 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:53.023326 [ 3014 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:53.084055 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:53.118879 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:53.190945 [ 3024 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:53.309843 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:53.419995 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:53.843206 [ 3044 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:53.896054 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:53.937664 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:54.009675 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:54.031040 [ 3047 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:54.195028 [ 3052 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:54.525941 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:54.566953 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:54.846476 [ 3067 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:55.034182 [ 3066 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:55.101053 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:55.198228 [ 3076 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:55.716691 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:55.754544 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:55.849133 [ 3084 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:55.873922 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:56.036275 [ 3090 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:56.200857 [ 3082 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:56.421342 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:56.465847 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:56.497972 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:56.590925 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:56.851694 [ 3115 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:56.884808 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:57.038170 [ 3131 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:57.043388 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:57.065280 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:57.211887 [ 3127 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:57.282035 [ 13129 ] {0e4618e9-edbe-4048-9a0d-c12b7e8ff635} executeQuery: Code: 69. DB::Exception: Decimal value is too big: 15 digits were read: 123456789123451e-1. Expected to read decimal with scale 5 and precision 18: In scope SELECT CAST('12345678912345.1', 'Decimal(10, 5)'). (ARGUMENT_OUT_OF_BOUND) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46022) (comment: 01327_decimal_cut_extra_digits_after_point.sql) (in query: -- But it's just Decimal64, so there is the limit. SELECT CAST('12345678912345.1' AS Decimal(10, 5));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception&, int&, unsigned int&, unsigned int&>(int, FormatStringHelperImpl::type, std::type_identity&>::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, unsigned int&, DB::Decimal&, int&, unsigned int&, unsigned int&) @ 0x00000000279d16d5 4. ./src/IO/readDecimalText.h:172: void DB::readDecimalText, void>(DB::ReadBuffer&, DB::Decimal&, unsigned int, unsigned int&, bool) @ 0x00000000279ccd5c 5. ./build_docker/./src/DataTypes/Serializations/SerializationDecimal.cpp:48: DB::SerializationDecimal>::readText(DB::Decimal&, DB::ReadBuffer&, unsigned int, unsigned int, bool) @ 0x0000000027adfe54 6. COW::immutable_ptr DB::(anonymous namespace)::ConvertImpl>, DB::(anonymous namespace)::FunctionCastName, (DB::FormatSettings::DateTimeOverflowBehavior)0>::execute(std::vector> const&, std::shared_ptr const&, unsigned long, DB::(anonymous namespace)::BehaviourOnErrorFromString, unsigned int) @ 0x000000000b03625a 7. _ZN2DBL22callOnIndexAndDataTypeINS_15DataTypeDecimalINS_7DecimalIlEEEEZZNKS_12_GLOBAL__N_112FunctionCast20createDecimalWrapperIS4_Q17IsDataTypeDecimalIT_EEENSt3__18functionIFN3COWINS_7IColumnEE13immutable_ptrISC_EERNS9_6vectorINS_21ColumnWithTypeAndNameENS9_9allocatorISH_EEEERKNS9_10shared_ptrIKNS_9IDataTypeEEEPKNS_14ColumnNullableEmEEESR_PKS8_bENKUlSL_SR_SU_mE_clESL_SR_SU_mEUlRSX_E_JEEEbNS_9TypeIndexEOT0_DpOT1_ @ 0x000000000b0160ee 8. _ZNSt3__110__function16__policy_invokerIFN3COWIN2DB7IColumnEE13immutable_ptrIS4_EERNS_6vectorINS3_21ColumnWithTypeAndNameENS_9allocatorIS9_EEEERKNS_10shared_ptrIKNS3_9IDataTypeEEEPKNS3_14ColumnNullableEmEE11__call_implINS0_20__default_alloc_funcIZNKS3_12_GLOBAL__N_112FunctionCast20createDecimalWrapperINS3_15DataTypeDecimalINS3_7DecimalIlEEEEQ17IsDataTypeDecimalIT_EEENS_8functionISN_EESJ_PKSY_bEUlSD_SJ_SM_mE_SN_EEEES7_PKNS0_16__policy_storageESD_SJ_SM_m @ 0x000000000b011993 9. DB::(anonymous namespace)::ExecutableFunctionCast::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000acf2e8b 10. DB::IExecutableFunction::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aaff0b5 11. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 12. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 13. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 14. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce7e4 15. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 16. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 20. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 21. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 22. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 23. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 24. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 25. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 26. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 27. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 29. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 30. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 31. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 2025.09.25 21:22:57.286344 [ 13129 ] {0e4618e9-edbe-4048-9a0d-c12b7e8ff635} TCPHandler: Code: 69. DB::Exception: Decimal value is too big: 15 digits were read: 123456789123451e-1. Expected to read decimal with scale 5 and precision 18: In scope SELECT CAST('12345678912345.1', 'Decimal(10, 5)'). (ARGUMENT_OUT_OF_BOUND), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception&, int&, unsigned int&, unsigned int&>(int, FormatStringHelperImpl::type, std::type_identity&>::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, unsigned int&, DB::Decimal&, int&, unsigned int&, unsigned int&) @ 0x00000000279d16d5 4. ./src/IO/readDecimalText.h:172: void DB::readDecimalText, void>(DB::ReadBuffer&, DB::Decimal&, unsigned int, unsigned int&, bool) @ 0x00000000279ccd5c 5. ./build_docker/./src/DataTypes/Serializations/SerializationDecimal.cpp:48: DB::SerializationDecimal>::readText(DB::Decimal&, DB::ReadBuffer&, unsigned int, unsigned int, bool) @ 0x0000000027adfe54 6. COW::immutable_ptr DB::(anonymous namespace)::ConvertImpl>, DB::(anonymous namespace)::FunctionCastName, (DB::FormatSettings::DateTimeOverflowBehavior)0>::execute(std::vector> const&, std::shared_ptr const&, unsigned long, DB::(anonymous namespace)::BehaviourOnErrorFromString, unsigned int) @ 0x000000000b03625a 7. _ZN2DBL22callOnIndexAndDataTypeINS_15DataTypeDecimalINS_7DecimalIlEEEEZZNKS_12_GLOBAL__N_112FunctionCast20createDecimalWrapperIS4_Q17IsDataTypeDecimalIT_EEENSt3__18functionIFN3COWINS_7IColumnEE13immutable_ptrISC_EERNS9_6vectorINS_21ColumnWithTypeAndNameENS9_9allocatorISH_EEEERKNS9_10shared_ptrIKNS_9IDataTypeEEEPKNS_14ColumnNullableEmEEESR_PKS8_bENKUlSL_SR_SU_mE_clESL_SR_SU_mEUlRSX_E_JEEEbNS_9TypeIndexEOT0_DpOT1_ @ 0x000000000b0160ee 8. _ZNSt3__110__function16__policy_invokerIFN3COWIN2DB7IColumnEE13immutable_ptrIS4_EERNS_6vectorINS3_21ColumnWithTypeAndNameENS_9allocatorIS9_EEEERKNS_10shared_ptrIKNS3_9IDataTypeEEEPKNS3_14ColumnNullableEmEE11__call_implINS0_20__default_alloc_funcIZNKS3_12_GLOBAL__N_112FunctionCast20createDecimalWrapperINS3_15DataTypeDecimalINS3_7DecimalIlEEEEQ17IsDataTypeDecimalIT_EEENS_8functionISN_EESJ_PKSY_bEUlSD_SJ_SM_mE_SN_EEEES7_PKNS0_16__policy_storageESD_SJ_SM_m @ 0x000000000b011993 9. DB::(anonymous namespace)::ExecutableFunctionCast::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000acf2e8b 10. DB::IExecutableFunction::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aaff0b5 11. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 12. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 13. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 14. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce7e4 15. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 16. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 20. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 21. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 22. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 23. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 24. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 25. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 26. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 27. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 28. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 29. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 30. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 31. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 2025.09.25 21:22:57.409918 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:57.420821 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:57.567803 [ 10537 ] {1bb2600b-9d1b-4e87-8087-398667eb91ca} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=733). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38686) (comment: 01676_reinterpret_as.sql) (in query: SELECT reinterpret(49, 'FixedString(4)');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:57.569623 [ 10537 ] {1bb2600b-9d1b-4e87-8087-398667eb91ca} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=733). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:22:57.585198 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:57.855149 [ 3146 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:58.040400 [ 3147 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:58.213960 [ 3155 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:58.565613 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:58.607079 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:58.682829 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:58.858373 [ 3176 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:59.014476 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:59.043371 [ 3181 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:59.217283 [ 3185 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:59.699503 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:22:59.861396 [ 3199 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:00.046236 [ 3192 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:00.057833 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:00.078576 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:00.099297 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:00.220027 [ 3205 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:00.677725 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:00.864674 [ 3220 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:00.895319 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:00.913214 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:01.049107 [ 3218 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:01.223055 [ 3227 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:01.574427 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:01.868047 [ 3250 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:02.052438 [ 3235 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:02.147136 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:02.167556 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:02.226271 [ 2745 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:02.358395 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:02.588911 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:02.742053 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:02.743991 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:02.871669 [ 2767 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:02.887755 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:03.056142 [ 2763 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:03.229178 [ 2769 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:03.258530 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:03.875207 [ 2783 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:04.059847 [ 2782 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:04.150306 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:04.180276 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:04.232580 [ 2750 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:04.368878 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:04.878597 [ 2801 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:04.918489 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:04.980645 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:05.063284 [ 2796 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:05.235233 [ 2805 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:05.376315 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:05.396693 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:05.544863 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:05.676298 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:05.882255 [ 2831 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:05.991517 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:06.066812 [ 2843 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:06.237977 [ 2848 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:06.305862 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:06.357059 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:06.688071 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:06.712792 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:06.885155 [ 2872 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:06.971007 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:07.070554 [ 2877 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:07.240943 [ 2874 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:07.596076 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:07.622457 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:07.734005 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:07.821476 [ 13129 ] {e7fe2235-6830-4040-b627-1593629e5027} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38720) (comment: 00409_shard_limit_by.sql) (in query: -- LIMIT BY doesn't affect result of GROUP BY SELECT Num, count(*) FROM limit_by GROUP BY Num ORDER BY Num LIMIT 2 BY Num;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:07.828735 [ 13129 ] {e7fe2235-6830-4040-b627-1593629e5027} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007f12cd75eac3 14. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:07.888503 [ 2902 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:08.074039 [ 2904 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:08.244485 [ 2905 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:08.295783 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:08.340265 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:08.375328 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:08.734307 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:08.836763 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:08.891401 [ 2920 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:08.892775 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:09.083199 [ 2925 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:09.234166 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:09.247548 [ 2944 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:09.320651 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:09.716874 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:09.894386 [ 2951 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:10.086023 [ 2961 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:10.086611 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:10.115571 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:10.200316 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:10.250871 [ 2965 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:10.897449 [ 2966 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:11.089013 [ 2976 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:11.253635 [ 2975 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:11.287370 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:11.301425 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:11.342408 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:11.578314 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:11.965572 [ 3000 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:12.100301 [ 2998 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:12.256337 [ 3005 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:12.492132 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:12.587419 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:12.614882 [ 13129 ] {} TCPHandler: Code: 452. DB::Exception: Setting allow_experimental_object_type should not be changed. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/remove_if.h:32: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f80932 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./build_docker/./src/Server/TCPHandler.cpp:2034: DB::TCPHandler::receiveQuery() @ 0x000000002d1a17ad 10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::receivePacket() @ 0x000000002d1877e2 11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d175061 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:12.786467 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:12.967585 [ 3025 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:13.103457 [ 3019 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:13.215695 [ 13129 ] {722da39f-2ac9-44ad-ad68-20c038d06425} executeQuery: Code: 60. DB::Exception: Table test_3.t_json_15 does not exist. (UNKNOWN_TABLE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60410) (comment: 01825_type_json_15.sh) (in query: INSERT INTO t_json_15 FORMAT JSONAsObject), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr) const @ 0x0000000027dd1ade 5. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr, std::optional*) const @ 0x00000000286af00b 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:801: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr) const @ 0x00000000286bb117 7. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:0: DB::InterpreterInsertQuery::getTable(DB::ASTInsertQuery&) @ 0x000000002934b836 8. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:748: DB::InterpreterInsertQuery::execute() @ 0x000000002935ad49 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:13.220547 [ 13129 ] {722da39f-2ac9-44ad-ad68-20c038d06425} TCPHandler: Code: 60. DB::Exception: Table test_3.t_json_15 does not exist. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr) const @ 0x0000000027dd1ade 5. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr, std::optional*) const @ 0x00000000286af00b 6. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:801: DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr) const @ 0x00000000286bb117 7. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:0: DB::InterpreterInsertQuery::getTable(DB::ASTInsertQuery&) @ 0x000000002934b836 8. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:748: DB::InterpreterInsertQuery::execute() @ 0x000000002935ad49 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:13.259813 [ 3016 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:13.594037 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:13.814432 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:13.860895 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:13.876179 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:13.901537 [ 13129 ] {efac3d91-527a-4730-9dcd-fe99bfa71314} executeQuery: Code: 60. DB::Exception: Unknown table expression identifier 't_json_15' in scope SELECT DISTINCT toTypeName(obj) FROM t_json_15. (UNKNOWN_TABLE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60414) (comment: 01825_type_json_15.sh) (in query: SELECT DISTINCT toTypeName(obj) FROM t_json_15;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:13.918228 [ 13129 ] {efac3d91-527a-4730-9dcd-fe99bfa71314} TCPHandler: Code: 60. DB::Exception: Unknown table expression identifier 't_json_15' in scope SELECT DISTINCT toTypeName(obj) FROM t_json_15. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:13.969874 [ 3050 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:14.105952 [ 3059 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:14.176457 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:14.233475 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:14.257352 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:14.267838 [ 3058 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:14.414991 [ 13129 ] {548fcbfe-88fe-4dc3-8419-506b9c319a9d} executeQuery: Code: 60. DB::Exception: Unknown table expression identifier 't_json_15' in scope SELECT obj FROM t_json_15 ORDER BY obj.id ASC. (UNKNOWN_TABLE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60416) (comment: 01825_type_json_15.sh) (in query: SELECT obj FROM t_json_15 ORDER BY obj.id FORMAT JSONEachRow), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:14.419356 [ 13129 ] {548fcbfe-88fe-4dc3-8419-506b9c319a9d} TCPHandler: Code: 60. DB::Exception: Unknown table expression identifier 't_json_15' in scope SELECT obj FROM t_json_15 ORDER BY obj.id ASC. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:15.009514 [ 13129 ] {2620be09-0f7e-41f9-8a13-84b55f60633c} executeQuery: Code: 60. DB::Exception: Unknown table expression identifier 't_json_15' in scope SELECT obj.key_0.key_1.key_2, obj.key_0.key_1.key_8, obj.key_0.key_10, obj.key_0.key_0 FROM t_json_15 ORDER BY obj.id ASC. (UNKNOWN_TABLE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60428) (comment: 01825_type_json_15.sh) (in query: SELECT obj.key_0.key_1.key_2, obj.key_0.key_1.key_8, obj.key_0.key_10, obj.key_0.key_0 FROM t_json_15 ORDER BY obj.id), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:15.024174 [ 13129 ] {2620be09-0f7e-41f9-8a13-84b55f60633c} TCPHandler: Code: 60. DB::Exception: Unknown table expression identifier 't_json_15' in scope SELECT obj.key_0.key_1.key_2, obj.key_0.key_1.key_8, obj.key_0.key_10, obj.key_0.key_0 FROM t_json_15 ORDER BY obj.id ASC. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4229: DB::QueryAnalyzer::initializeQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b8590 5. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5417: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835acd8 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 8. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 11. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 12. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 16. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 20. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 21. asan_thread_start(void*) @ 0x000000000aa43059 22. ? @ 0x00007f12cd75eac3 23. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:15.040457 [ 3089 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:15.107427 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:15.108887 [ 3083 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:15.264610 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:15.270677 [ 3093 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:15.294479 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:15.394455 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:15.680390 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:15.702891 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:15.731962 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:15.789864 [ 13129 ] {969e14d9-eec7-4672-8449-8d075edd2cd4} executeQuery: Code: 60. DB::Exception: Table test_3.t_json_15 does not exist. (UNKNOWN_TABLE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60440) (comment: 01825_type_json_15.sh) (in query: DROP TABLE t_json_15;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr) const @ 0x0000000027dd1ade 5. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr, std::optional*) const @ 0x00000000286af00b 6. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:969: DB::DatabaseCatalog::getDatabaseAndTable(DB::StorageID const&, std::shared_ptr) const @ 0x00000000286bb62c 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291eeb31 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 14. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 18. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:15.796278 [ 13129 ] {969e14d9-eec7-4672-8449-8d075edd2cd4} TCPHandler: Code: 60. DB::Exception: Table test_3.t_json_15 does not exist. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Databases/IDatabase.cpp:37: DB::IDatabase::getTable(String const&, std::shared_ptr) const @ 0x0000000027dd1ade 5. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr, std::optional*) const @ 0x00000000286af00b 6. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:969: DB::DatabaseCatalog::getDatabaseAndTable(DB::StorageID const&, std::shared_ptr) const @ 0x00000000286bb62c 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291eeb31 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 14. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 18. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:16.042861 [ 3104 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:16.111901 [ 3110 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:16.120728 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:16.273765 [ 3122 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:16.513384 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:16.546334 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:16.798283 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:16.960035 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:16.984524 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:17.045324 [ 3136 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:17.114250 [ 3150 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:17.276438 [ 3147 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:17.477879 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:17.494475 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:17.607077 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:18.048793 [ 3161 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:18.065622 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:18.099212 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:18.117126 [ 3178 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:18.279655 [ 3168 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:18.643284 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:18.677427 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:18.688828 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:19.051661 [ 3197 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:19.124973 [ 3199 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:19.219180 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:19.245692 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:19.282500 [ 3192 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:19.693644 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:20.060299 [ 3209 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:20.127981 [ 3213 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:20.279459 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:20.285637 [ 3208 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:20.293524 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:20.495328 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:20.619265 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:20.640290 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:20.763228 [ 10537 ] {09c4ce52-e6d5-4e89-9a63-8df0fbf9a18c} executeQuery: Code: 660. DB::Exception: Unable to connect to HDFS: Hdfs::HdfsRpcException: HdfsFailoverException: Failed to invoke RPC call "getFsStats" on server "localhost:12222" Caused by: HdfsNetworkConnectException: Connect to "localhost:12222" failed: (errno: 111) Connection refused. (HDFS_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:38704) (comment: 02845_table_function_hdfs_filter_by_virtual_columns.sh) (in query: insert into table function hdfs('hdfs://localhost:12222/02845_table_function_hdfs_filter_by_virtual_columns_test_5.data1.tsv') select 1 settings hdfs_truncate_on_insert=1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Storages/ObjectStorage/HDFS/HDFSCommon.cpp:172: DB::createHDFSFS(hdfsBuilder*) @ 0x000000002688a7ef 5. ./build_docker/./src/Disks/ObjectStorages/HDFS/HDFSObjectStorage.cpp:35: DB::HDFSObjectStorage::initializeHDFSFS() const @ 0x0000000026ba3602 6. ./contrib/llvm-project/libcxx/include/optional:344: DB::HDFSObjectStorage::writeObject(DB::StoredObject const&, DB::WriteMode, std::optional, std::allocator>>>, unsigned long, DB::WriteSettings const&) @ 0x0000000026ba6fe7 7. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSink.cpp:0: DB::StorageObjectStorageSink::StorageObjectStorageSink(std::shared_ptr, std::shared_ptr, std::optional const&, DB::Block const&, std::shared_ptr, String const&) @ 0x00000000267d3706 8. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::StorageObjectStorageSink* std::construct_at[abi:v15007] const&, std::shared_ptr, std::optional const&, DB::Block const&, std::shared_ptr&, DB::StorageObjectStorageSink*>(DB::StorageObjectStorageSink*, std::shared_ptr const&, std::shared_ptr&&, std::optional const&, DB::Block const&, std::shared_ptr&) @ 0x00000000267c64b8 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageObjectStorage::write(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, bool) @ 0x00000000267b95fb 10. ./build_docker/./src/Processors/Transforms/buildPushingToViewsChain.cpp:573: DB::buildPushingToViewsChain(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr const&, bool, std::shared_ptr, std::shared_ptr, std::atomic*, bool, DB::Block const&) @ 0x000000002db6fb52 11. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:342: DB::InterpreterInsertQuery::buildSink(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::atomic*) @ 0x000000002935079f 12. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:427: DB::InterpreterInsertQuery::buildPreAndSinkChains(unsigned long, unsigned long, std::shared_ptr, std::shared_ptr const&, DB::Block const&) @ 0x0000000029352541 13. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:625: DB::InterpreterInsertQuery::buildInsertSelectPipeline(DB::ASTInsertQuery&, std::shared_ptr) @ 0x0000000029355dd4 14. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:789: DB::InterpreterInsertQuery::execute() @ 0x000000002935b5a0 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:20.769057 [ 10537 ] {09c4ce52-e6d5-4e89-9a63-8df0fbf9a18c} TCPHandler: Code: 660. DB::Exception: Unable to connect to HDFS: Hdfs::HdfsRpcException: HdfsFailoverException: Failed to invoke RPC call "getFsStats" on server "localhost:12222" Caused by: HdfsNetworkConnectException: Connect to "localhost:12222" failed: (errno: 111) Connection refused. (HDFS_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Storages/ObjectStorage/HDFS/HDFSCommon.cpp:172: DB::createHDFSFS(hdfsBuilder*) @ 0x000000002688a7ef 5. ./build_docker/./src/Disks/ObjectStorages/HDFS/HDFSObjectStorage.cpp:35: DB::HDFSObjectStorage::initializeHDFSFS() const @ 0x0000000026ba3602 6. ./contrib/llvm-project/libcxx/include/optional:344: DB::HDFSObjectStorage::writeObject(DB::StoredObject const&, DB::WriteMode, std::optional, std::allocator>>>, unsigned long, DB::WriteSettings const&) @ 0x0000000026ba6fe7 7. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSink.cpp:0: DB::StorageObjectStorageSink::StorageObjectStorageSink(std::shared_ptr, std::shared_ptr, std::optional const&, DB::Block const&, std::shared_ptr, String const&) @ 0x00000000267d3706 8. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::StorageObjectStorageSink* std::construct_at[abi:v15007] const&, std::shared_ptr, std::optional const&, DB::Block const&, std::shared_ptr&, DB::StorageObjectStorageSink*>(DB::StorageObjectStorageSink*, std::shared_ptr const&, std::shared_ptr&&, std::optional const&, DB::Block const&, std::shared_ptr&) @ 0x00000000267c64b8 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageObjectStorage::write(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, bool) @ 0x00000000267b95fb 10. ./build_docker/./src/Processors/Transforms/buildPushingToViewsChain.cpp:573: DB::buildPushingToViewsChain(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr const&, bool, std::shared_ptr, std::shared_ptr, std::atomic*, bool, DB::Block const&) @ 0x000000002db6fb52 11. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:342: DB::InterpreterInsertQuery::buildSink(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::atomic*) @ 0x000000002935079f 12. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:427: DB::InterpreterInsertQuery::buildPreAndSinkChains(unsigned long, unsigned long, std::shared_ptr, std::shared_ptr const&, DB::Block const&) @ 0x0000000029352541 13. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:625: DB::InterpreterInsertQuery::buildInsertSelectPipeline(DB::ASTInsertQuery&, std::shared_ptr) @ 0x0000000029355dd4 14. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:789: DB::InterpreterInsertQuery::execute() @ 0x000000002935b5a0 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:21.063599 [ 3256 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:21.130543 [ 3246 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:21.288670 [ 3245 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:21.487070 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:21.691883 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:21.856492 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:21.909524 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:22.070087 [ 2763 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:22.133399 [ 2773 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:22.291597 [ 2780 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:22.431789 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:22.456523 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:22.495223 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:22.850379 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:23.072927 [ 2787 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:23.135780 [ 2795 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:23.293944 [ 2800 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:23.426232 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:23.761112 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:23.787996 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:24.075738 [ 2816 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:24.145302 [ 2814 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:24.297310 [ 2828 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:24.478932 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:24.689965 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:24.708153 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:24.935721 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:25.078537 [ 2843 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:25.147718 [ 2844 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:25.300604 [ 2841 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:25.654388 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:25.680165 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:26.058724 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:26.082270 [ 2847 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:26.150586 [ 2863 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:26.172455 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:26.209258 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:26.303645 [ 2865 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:26.623345 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:26.660470 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:26.705292 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:26.963367 [ 13110 ] {acec9496-aba4-4bd2-9ba4-3fe3e3777253} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60388) (comment: 02447_drop_database_replica.sh) (in query: drop database rdb_test_t1fje8px), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:26.979801 [ 13110 ] {acec9496-aba4-4bd2-9ba4-3fe3e3777253} TCPHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:27.085022 [ 2880 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:27.156069 [ 2888 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:27.306928 [ 2891 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:27.408806 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:27.501299 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:27.746575 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:28.088098 [ 2910 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:28.159043 [ 2894 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:28.173727 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:28.309675 [ 2917 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:28.329623 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:28.402612 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:28.466983 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:28.722343 [ 10192 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:28.910877 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:29.027550 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:29.075369 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:29.090636 [ 2942 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:29.161804 [ 2943 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:29.288243 [ 3363 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {f2eeeeb5-70de-4a33-835e-3bff489e4be8::202509_1_117_28}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001adfc17a 4. ./build_docker/./src/Common/MemoryTracker.cpp:187: MemoryTracker::injectFault() const @ 0x000000001adf8d1a 5. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x000000002cc30a65 6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f47b 7. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:29.312252 [ 2950 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:29.492377 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:29.756819 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:29.771506 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:29.940341 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:30.092993 [ 2970 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:30.168276 [ 2968 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:30.314874 [ 2991 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:30.654047 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:30.855622 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:30.879114 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:31.097101 [ 3015 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:31.171605 [ 3004 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:31.317939 [ 3013 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:31.480598 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:31.883894 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:32.103421 [ 3034 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:32.109328 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:32.131313 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:32.175213 [ 3038 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:32.321161 [ 3044 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:32.712671 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:32.734467 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:32.795562 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:33.106637 [ 3063 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:33.183220 [ 3060 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:33.235115 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:33.270510 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:33.324142 [ 3062 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:33.477935 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:33.876525 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:34.050867 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:34.109742 [ 3088 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:34.118429 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:34.185859 [ 3099 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:34.327561 [ 3095 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:34.863086 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:34.979211 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:34.997434 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:35.078561 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:35.113125 [ 3122 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:35.189617 [ 3115 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:35.331138 [ 3121 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:35.509334 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:35.935310 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:35.955217 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:35.957324 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:36.119161 [ 3151 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:36.193213 [ 3136 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:36.334627 [ 3150 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:36.393368 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:36.565925 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:36.592069 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:37.122459 [ 3178 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:37.175368 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:37.195519 [ 3175 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:37.337523 [ 3182 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:37.857083 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:37.912603 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:38.027183 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:38.125712 [ 3201 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:38.198437 [ 3169 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:38.282757 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:38.340851 [ 3206 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:38.450973 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:38.485769 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:38.766025 [ 13130 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:38.981095 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:39.129405 [ 3236 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:39.201985 [ 3233 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:39.299165 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:39.347403 [ 3240 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:39.348062 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:39.413177 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:39.867047 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:40.135045 [ 3255 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:40.205162 [ 3260 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:40.307738 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:40.350047 [ 3259 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:40.543756 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:40.578941 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:41.137802 [ 2769 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:41.208355 [ 2755 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:41.231974 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:41.283370 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:41.352696 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:41.354364 [ 2774 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:41.949424 [ 13129 ] {b0f510de-3354-435a-aff3-01f90098db59} executeQuery: Code: 660. DB::Exception: Unable to connect to HDFS: Hdfs::HdfsRpcException: HdfsFailoverException: Failed to invoke RPC call "getFsStats" on server "localhost:12222" Caused by: HdfsNetworkConnectException: Connect to "localhost:12222" failed: (errno: 111) Connection refused. (HDFS_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:35440) (comment: 02845_table_function_hdfs_filter_by_virtual_columns.sh) (in query: insert into table function hdfs('hdfs://localhost:12222/02845_table_function_hdfs_filter_by_virtual_columns_test_5.data2.tsv') select 2 settings hdfs_truncate_on_insert=1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Storages/ObjectStorage/HDFS/HDFSCommon.cpp:172: DB::createHDFSFS(hdfsBuilder*) @ 0x000000002688a7ef 5. ./build_docker/./src/Disks/ObjectStorages/HDFS/HDFSObjectStorage.cpp:35: DB::HDFSObjectStorage::initializeHDFSFS() const @ 0x0000000026ba3602 6. ./contrib/llvm-project/libcxx/include/optional:344: DB::HDFSObjectStorage::writeObject(DB::StoredObject const&, DB::WriteMode, std::optional, std::allocator>>>, unsigned long, DB::WriteSettings const&) @ 0x0000000026ba6fe7 7. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSink.cpp:0: DB::StorageObjectStorageSink::StorageObjectStorageSink(std::shared_ptr, std::shared_ptr, std::optional const&, DB::Block const&, std::shared_ptr, String const&) @ 0x00000000267d3706 8. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::StorageObjectStorageSink* std::construct_at[abi:v15007] const&, std::shared_ptr, std::optional const&, DB::Block const&, std::shared_ptr&, DB::StorageObjectStorageSink*>(DB::StorageObjectStorageSink*, std::shared_ptr const&, std::shared_ptr&&, std::optional const&, DB::Block const&, std::shared_ptr&) @ 0x00000000267c64b8 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageObjectStorage::write(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, bool) @ 0x00000000267b95fb 10. ./build_docker/./src/Processors/Transforms/buildPushingToViewsChain.cpp:573: DB::buildPushingToViewsChain(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr const&, bool, std::shared_ptr, std::shared_ptr, std::atomic*, bool, DB::Block const&) @ 0x000000002db6fb52 11. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:342: DB::InterpreterInsertQuery::buildSink(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::atomic*) @ 0x000000002935079f 12. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:427: DB::InterpreterInsertQuery::buildPreAndSinkChains(unsigned long, unsigned long, std::shared_ptr, std::shared_ptr const&, DB::Block const&) @ 0x0000000029352541 13. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:625: DB::InterpreterInsertQuery::buildInsertSelectPipeline(DB::ASTInsertQuery&, std::shared_ptr) @ 0x0000000029355dd4 14. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:789: DB::InterpreterInsertQuery::execute() @ 0x000000002935b5a0 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:41.966756 [ 13129 ] {b0f510de-3354-435a-aff3-01f90098db59} TCPHandler: Code: 660. DB::Exception: Unable to connect to HDFS: Hdfs::HdfsRpcException: HdfsFailoverException: Failed to invoke RPC call "getFsStats" on server "localhost:12222" Caused by: HdfsNetworkConnectException: Connect to "localhost:12222" failed: (errno: 111) Connection refused. (HDFS_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Storages/ObjectStorage/HDFS/HDFSCommon.cpp:172: DB::createHDFSFS(hdfsBuilder*) @ 0x000000002688a7ef 5. ./build_docker/./src/Disks/ObjectStorages/HDFS/HDFSObjectStorage.cpp:35: DB::HDFSObjectStorage::initializeHDFSFS() const @ 0x0000000026ba3602 6. ./contrib/llvm-project/libcxx/include/optional:344: DB::HDFSObjectStorage::writeObject(DB::StoredObject const&, DB::WriteMode, std::optional, std::allocator>>>, unsigned long, DB::WriteSettings const&) @ 0x0000000026ba6fe7 7. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSink.cpp:0: DB::StorageObjectStorageSink::StorageObjectStorageSink(std::shared_ptr, std::shared_ptr, std::optional const&, DB::Block const&, std::shared_ptr, String const&) @ 0x00000000267d3706 8. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::StorageObjectStorageSink* std::construct_at[abi:v15007] const&, std::shared_ptr, std::optional const&, DB::Block const&, std::shared_ptr&, DB::StorageObjectStorageSink*>(DB::StorageObjectStorageSink*, std::shared_ptr const&, std::shared_ptr&&, std::optional const&, DB::Block const&, std::shared_ptr&) @ 0x00000000267c64b8 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageObjectStorage::write(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, bool) @ 0x00000000267b95fb 10. ./build_docker/./src/Processors/Transforms/buildPushingToViewsChain.cpp:573: DB::buildPushingToViewsChain(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr const&, bool, std::shared_ptr, std::shared_ptr, std::atomic*, bool, DB::Block const&) @ 0x000000002db6fb52 11. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:342: DB::InterpreterInsertQuery::buildSink(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::atomic*) @ 0x000000002935079f 12. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:427: DB::InterpreterInsertQuery::buildPreAndSinkChains(unsigned long, unsigned long, std::shared_ptr, std::shared_ptr const&, DB::Block const&) @ 0x0000000029352541 13. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:625: DB::InterpreterInsertQuery::buildInsertSelectPipeline(DB::ASTInsertQuery&, std::shared_ptr) @ 0x0000000029355dd4 14. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:789: DB::InterpreterInsertQuery::execute() @ 0x000000002935b5a0 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:23:41.969761 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:42.140704 [ 2787 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:42.211877 [ 2795 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:42.217853 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:42.268002 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:42.358004 [ 2797 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:42.902735 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:43.093497 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:43.136218 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:43.145221 [ 2807 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:43.215044 [ 2821 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:43.361352 [ 2818 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:43.586061 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:43.614638 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:43.924442 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:44.148643 [ 2823 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:44.217809 [ 2849 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:44.353345 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:44.364610 [ 2856 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:44.382879 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:45.028217 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:45.151053 [ 2865 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:45.220733 [ 2859 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:45.367845 [ 2871 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:45.665223 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:45.693538 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:45.883031 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:46.153954 [ 2897 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:46.223768 [ 2901 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:46.369998 [ 2886 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:46.438062 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:46.521259 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:46.565329 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:46.857214 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:47.157211 [ 2924 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:47.227177 [ 2925 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:47.376978 [ 2929 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:47.651231 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:47.675864 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:47.838339 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:48.160338 [ 2948 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:48.194100 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:48.216242 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:48.230236 [ 2955 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:48.246269 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:48.380230 [ 2965 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:48.653086 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:49.163260 [ 2971 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:49.233669 [ 2968 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:49.368067 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:49.383326 [ 2986 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:49.455924 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:49.564162 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:50.008262 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:50.166878 [ 2998 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:50.237254 [ 2999 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:50.386555 [ 3007 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:50.717688 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:50.797678 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:50.934798 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:51.170162 [ 3019 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:51.239380 [ 3016 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:51.389656 [ 3029 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:51.485424 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:51.769488 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:51.809928 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:52.173539 [ 3046 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:52.278654 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:52.336111 [ 3052 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:52.393267 [ 3059 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:52.540965 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:52.586685 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:53.131406 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:53.176721 [ 3072 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:53.341093 [ 3074 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:53.396549 [ 3075 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:53.570818 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:53.612354 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:54.056077 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:54.179753 [ 3094 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:54.344037 [ 3092 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:54.399659 [ 3102 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:54.556943 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:54.604529 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:54.872902 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:54.920031 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:55.182922 [ 3121 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:55.346319 [ 3119 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:55.358038 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:55.404755 [ 3131 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:56.119024 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:56.186348 [ 3148 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:56.241508 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:56.303892 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:56.348971 [ 3147 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:56.407781 [ 3155 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:56.920722 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:57.065961 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:57.091658 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:57.191046 [ 3172 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:57.352466 [ 3177 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:57.411915 [ 3163 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:57.571352 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:57.626324 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:57.844035 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:57.918482 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:57.942384 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:58.169949 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:58.197286 [ 3186 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:58.207849 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:58.355531 [ 3202 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:58.415110 [ 3192 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:58.574998 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:58.601663 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:58.825348 [ 13130 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:58.961396 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:59.268958 [ 3217 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:59.358558 [ 3223 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:59.419070 [ 3220 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:59.457986 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:59.482319 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:23:59.977795 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:00.013802 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:00.020292 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:00.272330 [ 3249 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:00.361515 [ 3247 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:00.392821 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:00.421789 [ 3239 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:01.021989 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:01.066981 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:01.092863 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:01.276080 [ 2760 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:01.351034 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:01.363727 [ 2772 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:01.424012 [ 2766 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:01.441013 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:01.490837 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:02.049753 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:02.152294 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:02.199183 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:02.279464 [ 2751 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:02.366575 [ 2788 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:02.426801 [ 2792 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:02.681651 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:02.869662 [ 13129 ] {e16ea7e6-1961-4b7b-b804-7996f299981e} executeQuery: Code: 660. DB::Exception: Unable to connect to HDFS: Hdfs::HdfsRpcException: HdfsFailoverException: Failed to invoke RPC call "getFsStats" on server "localhost:12222" Caused by: HdfsNetworkConnectException: Connect to "localhost:12222" failed: (errno: 111) Connection refused. (HDFS_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42244) (comment: 02845_table_function_hdfs_filter_by_virtual_columns.sh) (in query: insert into table function hdfs('hdfs://localhost:12222/02845_table_function_hdfs_filter_by_virtual_columns_test_5.data3.tsv') select 3 settings hdfs_truncate_on_insert=1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Storages/ObjectStorage/HDFS/HDFSCommon.cpp:172: DB::createHDFSFS(hdfsBuilder*) @ 0x000000002688a7ef 5. ./build_docker/./src/Disks/ObjectStorages/HDFS/HDFSObjectStorage.cpp:35: DB::HDFSObjectStorage::initializeHDFSFS() const @ 0x0000000026ba3602 6. ./contrib/llvm-project/libcxx/include/optional:344: DB::HDFSObjectStorage::writeObject(DB::StoredObject const&, DB::WriteMode, std::optional, std::allocator>>>, unsigned long, DB::WriteSettings const&) @ 0x0000000026ba6fe7 7. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSink.cpp:0: DB::StorageObjectStorageSink::StorageObjectStorageSink(std::shared_ptr, std::shared_ptr, std::optional const&, DB::Block const&, std::shared_ptr, String const&) @ 0x00000000267d3706 8. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::StorageObjectStorageSink* std::construct_at[abi:v15007] const&, std::shared_ptr, std::optional const&, DB::Block const&, std::shared_ptr&, DB::StorageObjectStorageSink*>(DB::StorageObjectStorageSink*, std::shared_ptr const&, std::shared_ptr&&, std::optional const&, DB::Block const&, std::shared_ptr&) @ 0x00000000267c64b8 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageObjectStorage::write(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, bool) @ 0x00000000267b95fb 10. ./build_docker/./src/Processors/Transforms/buildPushingToViewsChain.cpp:573: DB::buildPushingToViewsChain(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr const&, bool, std::shared_ptr, std::shared_ptr, std::atomic*, bool, DB::Block const&) @ 0x000000002db6fb52 11. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:342: DB::InterpreterInsertQuery::buildSink(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::atomic*) @ 0x000000002935079f 12. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:427: DB::InterpreterInsertQuery::buildPreAndSinkChains(unsigned long, unsigned long, std::shared_ptr, std::shared_ptr const&, DB::Block const&) @ 0x0000000029352541 13. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:625: DB::InterpreterInsertQuery::buildInsertSelectPipeline(DB::ASTInsertQuery&, std::shared_ptr) @ 0x0000000029355dd4 14. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:789: DB::InterpreterInsertQuery::execute() @ 0x000000002935b5a0 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:02.882191 [ 13129 ] {e16ea7e6-1961-4b7b-b804-7996f299981e} TCPHandler: Code: 660. DB::Exception: Unable to connect to HDFS: Hdfs::HdfsRpcException: HdfsFailoverException: Failed to invoke RPC call "getFsStats" on server "localhost:12222" Caused by: HdfsNetworkConnectException: Connect to "localhost:12222" failed: (errno: 111) Connection refused. (HDFS_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Storages/ObjectStorage/HDFS/HDFSCommon.cpp:172: DB::createHDFSFS(hdfsBuilder*) @ 0x000000002688a7ef 5. ./build_docker/./src/Disks/ObjectStorages/HDFS/HDFSObjectStorage.cpp:35: DB::HDFSObjectStorage::initializeHDFSFS() const @ 0x0000000026ba3602 6. ./contrib/llvm-project/libcxx/include/optional:344: DB::HDFSObjectStorage::writeObject(DB::StoredObject const&, DB::WriteMode, std::optional, std::allocator>>>, unsigned long, DB::WriteSettings const&) @ 0x0000000026ba6fe7 7. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSink.cpp:0: DB::StorageObjectStorageSink::StorageObjectStorageSink(std::shared_ptr, std::shared_ptr, std::optional const&, DB::Block const&, std::shared_ptr, String const&) @ 0x00000000267d3706 8. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::StorageObjectStorageSink* std::construct_at[abi:v15007] const&, std::shared_ptr, std::optional const&, DB::Block const&, std::shared_ptr&, DB::StorageObjectStorageSink*>(DB::StorageObjectStorageSink*, std::shared_ptr const&, std::shared_ptr&&, std::optional const&, DB::Block const&, std::shared_ptr&) @ 0x00000000267c64b8 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageObjectStorage::write(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, bool) @ 0x00000000267b95fb 10. ./build_docker/./src/Processors/Transforms/buildPushingToViewsChain.cpp:573: DB::buildPushingToViewsChain(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr const&, bool, std::shared_ptr, std::shared_ptr, std::atomic*, bool, DB::Block const&) @ 0x000000002db6fb52 11. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:342: DB::InterpreterInsertQuery::buildSink(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::atomic*) @ 0x000000002935079f 12. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:427: DB::InterpreterInsertQuery::buildPreAndSinkChains(unsigned long, unsigned long, std::shared_ptr, std::shared_ptr const&, DB::Block const&) @ 0x0000000029352541 13. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:625: DB::InterpreterInsertQuery::buildInsertSelectPipeline(DB::ASTInsertQuery&, std::shared_ptr) @ 0x0000000029355dd4 14. ./build_docker/./src/Interpreters/InterpreterInsertQuery.cpp:789: DB::InterpreterInsertQuery::execute() @ 0x000000002935b5a0 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:03.016609 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:03.133143 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:03.144646 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:03.282044 [ 2799 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:03.369338 [ 2810 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:03.429849 [ 2812 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:03.749517 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:03.941204 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:03.998067 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:04.285083 [ 2819 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:04.372662 [ 2831 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:04.432908 [ 2825 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:04.871152 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:04.932418 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:04.984143 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:05.288339 [ 2846 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:05.376193 [ 2842 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:05.436303 [ 2856 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:05.570413 [ 13130 ] {187d7c52-48c2-44e8-b7ea-dace5ea1285b} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:45296) (comment: 00900_orc_arrow_parquet_nested.sh) (in query: DROP DATABASE IF EXISTS test_qcrhbmc9 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:05.573755 [ 13130 ] {187d7c52-48c2-44e8-b7ea-dace5ea1285b} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:05.864104 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:06.042485 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:06.068171 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:06.291669 [ 2866 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:06.379257 [ 2868 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:06.439745 [ 2871 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:06.913673 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:07.072687 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:07.162387 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:07.295052 [ 2891 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:07.382481 [ 2886 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:07.445179 [ 2903 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:07.850701 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:07.866423 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:07.887275 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:08.327354 [ 2923 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:08.388382 [ 2934 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:08.448762 [ 2929 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:08.554940 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:08.807947 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:08.855200 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:09.058871 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:09.176163 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:09.330387 [ 2958 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:09.392116 [ 2949 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:09.452419 [ 2955 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:09.515714 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:09.941961 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:09.966557 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:10.333361 [ 2975 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:10.395219 [ 2979 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:10.417942 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:10.456082 [ 2984 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:11.012210 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:11.093339 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:11.337050 [ 3000 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:11.398433 [ 3005 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:11.459722 [ 3007 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:11.478439 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:11.499804 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:11.536888 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:12.147049 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:12.263590 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:12.341086 [ 3023 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:12.360289 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:12.389269 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:12.401202 [ 3033 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:12.463209 [ 3028 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:12.768462 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:12.964680 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:13.267089 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:13.343395 [ 3053 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:13.404339 [ 3050 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:13.466155 [ 3052 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:13.832669 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:13.883519 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:13.931134 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:14.347048 [ 3074 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:14.407256 [ 3078 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:14.468294 [ 3071 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:14.883996 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:14.953662 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:14.970633 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:15.350172 [ 3106 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:15.481960 [ 3107 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:15.483113 [ 3103 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:15.730946 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:15.755214 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:15.836027 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:16.286855 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:16.337480 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:16.290002 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:16.352566 [ 3126 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:16.485467 [ 3141 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:16.486633 [ 3138 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:16.516892 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:17.041669 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:17.058292 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:17.354876 [ 3143 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:17.459182 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:17.488598 [ 3157 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:17.490805 [ 3171 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:17.830648 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:17.840168 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:18.021812 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:18.357930 [ 3167 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:18.491819 [ 3097 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:18.493621 [ 3187 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:18.963919 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:18.992876 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:19.041128 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:19.361103 [ 3205 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:19.495232 [ 3217 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:19.496587 [ 3209 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:20.009684 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:20.087712 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:20.118565 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:20.363950 [ 3229 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:20.479363 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:20.498316 [ 3241 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:20.578937 [ 3240 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:20.583815 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:20.621989 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:20.988867 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:21.287261 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:21.309131 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:21.366017 [ 2747 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:21.501580 [ 2759 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:21.584140 [ 2749 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:21.601744 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:21.712276 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:21.725494 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:22.254744 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:22.380070 [ 2776 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:22.516417 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:22.537747 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:22.574802 [ 2770 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:22.586644 [ 2753 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:22.620901 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:23.116161 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:23.342469 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:23.360444 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:23.382785 [ 2798 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:23.577860 [ 2799 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:23.589774 [ 2810 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:23.812791 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:23.838798 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:23.851988 [ 13112 ] {82a676bf-e265-4374-ac1e-169face38789} executeQuery: Code: 660. DB::Exception: Unable to connect to HDFS: Hdfs::HdfsRpcException: HdfsFailoverException: Failed to invoke RPC call "getFsStats" on server "localhost:12222" Caused by: HdfsNetworkConnectException: Connect to "localhost:12222" failed: (errno: 111) Connection refused. (HDFS_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43028) (comment: 02845_table_function_hdfs_filter_by_virtual_columns.sh) (in query: select * from hdfs('hdfs://localhost:12222/02845_table_function_hdfs_filter_by_virtual_columns_test_5.data*.tsv', auto, 'x UInt64') where _file like '%data1%' format Null), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Storages/ObjectStorage/HDFS/HDFSCommon.cpp:172: DB::createHDFSFS(hdfsBuilder*) @ 0x000000002688a7ef 5. ./build_docker/./src/Disks/ObjectStorages/HDFS/HDFSObjectStorage.cpp:35: DB::HDFSObjectStorage::initializeHDFSFS() const @ 0x0000000026ba3602 6. ./build_docker/./src/Disks/ObjectStorages/HDFS/HDFSObjectStorage.cpp:189: DB::HDFSObjectStorage::listObjects(String const&, std::vector, std::allocator>>&, unsigned long) const @ 0x0000000026ba8c4d 7. ./contrib/llvm-project/libcxx/include/new:246: DB::IObjectStorage::iterate(String const&, unsigned long) const @ 0x000000002687a0bf 8. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSource.cpp:0: DB::StorageObjectStorageSource::GlobIterator::GlobIterator(std::shared_ptr, std::shared_ptr, DB::ActionsDAG::Node const*, DB::NamesAndTypesList const&, std::shared_ptr, std::vector, std::allocator>>*, unsigned long, bool, std::function) @ 0x00000000267e1ef3 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000267ecbb1 10. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSource.cpp:126: DB::StorageObjectStorageSource::createFileIterator(std::shared_ptr, DB::StorageObjectStorage::QuerySettings const&, std::shared_ptr, bool, std::shared_ptr const&, DB::ActionsDAG::Node const*, DB::NamesAndTypesList const&, std::vector, std::allocator>>*, std::function) @ 0x00000000267d96c9 11. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:233: DB::(anonymous namespace)::ReadFromObjectStorageStep::createIterator(DB::ActionsDAG::Node const*) @ 0x00000000267c14a8 12. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:172: DB::(anonymous namespace)::ReadFromObjectStorageStep::applyFilters(DB::ActionDAGNodes) @ 0x00000000267c0ef8 13. ./contrib/llvm-project/libcxx/include/vector:590: DB::SourceStepWithFilter::applyFilters() @ 0x000000002df93a35 14. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizePrimaryKeyConditionAndLimit.cpp:0: DB::QueryPlanOptimizations::optimizePrimaryKeyConditionAndLimit(std::vector> const&) @ 0x000000002df9366a 15. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:122: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list>&) @ 0x000000002df62d06 16. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:507: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x000000002de001ca 17. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:36: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddfec4b 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 20. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 21. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 23. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 27. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007f12cd75eac3 30. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:23.878740 [ 13112 ] {82a676bf-e265-4374-ac1e-169face38789} TCPHandler: Code: 660. DB::Exception: Unable to connect to HDFS: Hdfs::HdfsRpcException: HdfsFailoverException: Failed to invoke RPC call "getFsStats" on server "localhost:12222" Caused by: HdfsNetworkConnectException: Connect to "localhost:12222" failed: (errno: 111) Connection refused. (HDFS_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Storages/ObjectStorage/HDFS/HDFSCommon.cpp:172: DB::createHDFSFS(hdfsBuilder*) @ 0x000000002688a7ef 5. ./build_docker/./src/Disks/ObjectStorages/HDFS/HDFSObjectStorage.cpp:35: DB::HDFSObjectStorage::initializeHDFSFS() const @ 0x0000000026ba3602 6. ./build_docker/./src/Disks/ObjectStorages/HDFS/HDFSObjectStorage.cpp:189: DB::HDFSObjectStorage::listObjects(String const&, std::vector, std::allocator>>&, unsigned long) const @ 0x0000000026ba8c4d 7. ./contrib/llvm-project/libcxx/include/new:246: DB::IObjectStorage::iterate(String const&, unsigned long) const @ 0x000000002687a0bf 8. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSource.cpp:0: DB::StorageObjectStorageSource::GlobIterator::GlobIterator(std::shared_ptr, std::shared_ptr, DB::ActionsDAG::Node const*, DB::NamesAndTypesList const&, std::shared_ptr, std::vector, std::allocator>>*, unsigned long, bool, std::function) @ 0x00000000267e1ef3 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000267ecbb1 10. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSource.cpp:126: DB::StorageObjectStorageSource::createFileIterator(std::shared_ptr, DB::StorageObjectStorage::QuerySettings const&, std::shared_ptr, bool, std::shared_ptr const&, DB::ActionsDAG::Node const*, DB::NamesAndTypesList const&, std::vector, std::allocator>>*, std::function) @ 0x00000000267d96c9 11. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:233: DB::(anonymous namespace)::ReadFromObjectStorageStep::createIterator(DB::ActionsDAG::Node const*) @ 0x00000000267c14a8 12. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:172: DB::(anonymous namespace)::ReadFromObjectStorageStep::applyFilters(DB::ActionDAGNodes) @ 0x00000000267c0ef8 13. ./contrib/llvm-project/libcxx/include/vector:590: DB::SourceStepWithFilter::applyFilters() @ 0x000000002df93a35 14. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizePrimaryKeyConditionAndLimit.cpp:0: DB::QueryPlanOptimizations::optimizePrimaryKeyConditionAndLimit(std::vector> const&) @ 0x000000002df9366a 15. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:122: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list>&) @ 0x000000002df62d06 16. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:507: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x000000002de001ca 17. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:36: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddfec4b 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 20. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 21. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 23. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 27. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007f12cd75eac3 30. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:23.895130 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:24.385945 [ 2830 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:24.485290 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:24.520206 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:24.580192 [ 2834 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:24.592631 [ 2832 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:24.876665 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:24.877719 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:24.890018 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:25.188459 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:25.389567 [ 2852 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:25.583156 [ 2854 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:25.595817 [ 2862 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:25.968616 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:26.011443 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:26.039036 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:26.252812 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:26.284364 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:26.393084 [ 2871 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:26.538807 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:26.569840 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:26.586478 [ 2890 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:26.599164 [ 2883 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:26.932080 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:27.186614 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:27.212778 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:27.396429 [ 2906 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:27.589704 [ 2905 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:27.602053 [ 2916 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:27.694233 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:27.710893 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:27.766438 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:28.399636 [ 2922 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:28.593004 [ 2924 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:28.599248 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:28.604952 [ 2920 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:28.755090 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:28.797172 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:28.896782 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:28.921993 [ 10537 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:29.403107 [ 2937 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:29.522556 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:29.595899 [ 2936 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:29.608311 [ 2945 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:29.698917 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:29.821572 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:30.405856 [ 2962 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:30.600335 [ 2966 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:30.611618 [ 2973 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:30.745982 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:30.806727 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:30.977028 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:31.015781 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:31.156318 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:31.467358 [ 2982 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:31.602951 [ 2994 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:31.614055 [ 2981 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:31.841777 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:31.970450 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:32.081230 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:32.098610 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:32.470582 [ 3013 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:32.608147 [ 3009 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:32.617204 [ 3018 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:32.698483 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:33.024797 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:33.048394 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:33.132873 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:33.473838 [ 3036 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:33.585440 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:33.611576 [ 3042 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:33.620866 [ 3037 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:34.152737 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:34.174973 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:34.215112 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:34.476953 [ 3065 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:34.614976 [ 3054 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:34.624145 [ 3063 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:34.965231 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:35.054178 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:35.078388 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:35.480324 [ 3070 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:35.618092 [ 3071 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:35.627806 [ 3073 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:35.682068 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:35.881217 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:35.942663 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:36.485334 [ 3101 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:36.622512 [ 3111 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:36.631898 [ 3104 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:36.633842 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:36.908153 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:37.101231 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:37.120334 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:37.461915 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:37.488543 [ 3127 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:37.500176 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:37.625165 [ 3141 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:37.636870 [ 3138 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:37.798723 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:37.977259 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:38.492175 [ 3143 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:38.627514 [ 3157 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:38.639642 [ 3171 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:38.859195 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:38.894262 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:39.081928 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:39.495583 [ 3159 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:39.629974 [ 3189 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:39.642003 [ 3097 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:39.789407 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:39.803938 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:39.985319 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:40.498100 [ 3214 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:40.633064 [ 3216 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:40.645263 [ 3213 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:40.650191 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:40.682878 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:41.030752 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:41.218877 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:41.227337 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:41.500479 [ 3241 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:41.635950 [ 3243 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:41.636077 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:41.648294 [ 3238 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:42.087042 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:42.101924 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:42.502596 [ 3248 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:42.639256 [ 3255 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:42.651406 [ 2752 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:42.765757 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:42.994658 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:43.036655 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:43.357233 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:43.505517 [ 2772 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:43.642481 [ 2779 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:43.654304 [ 2773 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:43.677067 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:43.691520 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:44.069669 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:44.508919 [ 2751 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:44.529123 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:44.645176 [ 2783 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:44.657579 [ 2786 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:44.763391 [ 13112 ] {5449f1c0-4c8f-4abe-948c-dbd3b5e30b6a} executeQuery: Code: 660. DB::Exception: Unable to connect to HDFS: Hdfs::HdfsRpcException: HdfsFailoverException: Failed to invoke RPC call "getFsStats" on server "localhost:12222" Caused by: HdfsNetworkConnectException: Connect to "localhost:12222" failed: (errno: 111) Connection refused. (HDFS_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50724) (comment: 02845_table_function_hdfs_filter_by_virtual_columns.sh) (in query: select * from hdfs('hdfs://localhost:12222/02845_table_function_hdfs_filter_by_virtual_columns_test_5.data*.tsv', auto, 'x UInt64') where _path like '%data1%' format Null), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Storages/ObjectStorage/HDFS/HDFSCommon.cpp:172: DB::createHDFSFS(hdfsBuilder*) @ 0x000000002688a7ef 5. ./build_docker/./src/Disks/ObjectStorages/HDFS/HDFSObjectStorage.cpp:35: DB::HDFSObjectStorage::initializeHDFSFS() const @ 0x0000000026ba3602 6. ./build_docker/./src/Disks/ObjectStorages/HDFS/HDFSObjectStorage.cpp:189: DB::HDFSObjectStorage::listObjects(String const&, std::vector, std::allocator>>&, unsigned long) const @ 0x0000000026ba8c4d 7. ./contrib/llvm-project/libcxx/include/new:246: DB::IObjectStorage::iterate(String const&, unsigned long) const @ 0x000000002687a0bf 8. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSource.cpp:0: DB::StorageObjectStorageSource::GlobIterator::GlobIterator(std::shared_ptr, std::shared_ptr, DB::ActionsDAG::Node const*, DB::NamesAndTypesList const&, std::shared_ptr, std::vector, std::allocator>>*, unsigned long, bool, std::function) @ 0x00000000267e1ef3 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000267ecbb1 10. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSource.cpp:126: DB::StorageObjectStorageSource::createFileIterator(std::shared_ptr, DB::StorageObjectStorage::QuerySettings const&, std::shared_ptr, bool, std::shared_ptr const&, DB::ActionsDAG::Node const*, DB::NamesAndTypesList const&, std::vector, std::allocator>>*, std::function) @ 0x00000000267d96c9 11. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:233: DB::(anonymous namespace)::ReadFromObjectStorageStep::createIterator(DB::ActionsDAG::Node const*) @ 0x00000000267c14a8 12. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:172: DB::(anonymous namespace)::ReadFromObjectStorageStep::applyFilters(DB::ActionDAGNodes) @ 0x00000000267c0ef8 13. ./contrib/llvm-project/libcxx/include/vector:590: DB::SourceStepWithFilter::applyFilters() @ 0x000000002df93a35 14. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizePrimaryKeyConditionAndLimit.cpp:0: DB::QueryPlanOptimizations::optimizePrimaryKeyConditionAndLimit(std::vector> const&) @ 0x000000002df9366a 15. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:122: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list>&) @ 0x000000002df62d06 16. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:507: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x000000002de001ca 17. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:36: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddfec4b 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 20. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 21. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 23. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 27. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007f12cd75eac3 30. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:44.767868 [ 13112 ] {5449f1c0-4c8f-4abe-948c-dbd3b5e30b6a} TCPHandler: Code: 660. DB::Exception: Unable to connect to HDFS: Hdfs::HdfsRpcException: HdfsFailoverException: Failed to invoke RPC call "getFsStats" on server "localhost:12222" Caused by: HdfsNetworkConnectException: Connect to "localhost:12222" failed: (errno: 111) Connection refused. (HDFS_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000aad4b34 4. ./build_docker/./src/Storages/ObjectStorage/HDFS/HDFSCommon.cpp:172: DB::createHDFSFS(hdfsBuilder*) @ 0x000000002688a7ef 5. ./build_docker/./src/Disks/ObjectStorages/HDFS/HDFSObjectStorage.cpp:35: DB::HDFSObjectStorage::initializeHDFSFS() const @ 0x0000000026ba3602 6. ./build_docker/./src/Disks/ObjectStorages/HDFS/HDFSObjectStorage.cpp:189: DB::HDFSObjectStorage::listObjects(String const&, std::vector, std::allocator>>&, unsigned long) const @ 0x0000000026ba8c4d 7. ./contrib/llvm-project/libcxx/include/new:246: DB::IObjectStorage::iterate(String const&, unsigned long) const @ 0x000000002687a0bf 8. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSource.cpp:0: DB::StorageObjectStorageSource::GlobIterator::GlobIterator(std::shared_ptr, std::shared_ptr, DB::ActionsDAG::Node const*, DB::NamesAndTypesList const&, std::shared_ptr, std::vector, std::allocator>>*, unsigned long, bool, std::function) @ 0x00000000267e1ef3 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000267ecbb1 10. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSource.cpp:126: DB::StorageObjectStorageSource::createFileIterator(std::shared_ptr, DB::StorageObjectStorage::QuerySettings const&, std::shared_ptr, bool, std::shared_ptr const&, DB::ActionsDAG::Node const*, DB::NamesAndTypesList const&, std::vector, std::allocator>>*, std::function) @ 0x00000000267d96c9 11. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:233: DB::(anonymous namespace)::ReadFromObjectStorageStep::createIterator(DB::ActionsDAG::Node const*) @ 0x00000000267c14a8 12. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:172: DB::(anonymous namespace)::ReadFromObjectStorageStep::applyFilters(DB::ActionDAGNodes) @ 0x00000000267c0ef8 13. ./contrib/llvm-project/libcxx/include/vector:590: DB::SourceStepWithFilter::applyFilters() @ 0x000000002df93a35 14. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizePrimaryKeyConditionAndLimit.cpp:0: DB::QueryPlanOptimizations::optimizePrimaryKeyConditionAndLimit(std::vector> const&) @ 0x000000002df9366a 15. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:122: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list>&) @ 0x000000002df62d06 16. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:507: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x000000002de001ca 17. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:36: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddfec4b 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 20. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 21. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 23. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 27. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007f12cd75eac3 30. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:45.041614 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:45.086886 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:45.205198 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:45.435695 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:45.456212 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:45.507027 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:45.511388 [ 2810 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:45.647293 [ 2805 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:45.659921 [ 2815 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:46.167299 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:46.409707 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:46.432863 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:46.514331 [ 2836 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:46.631226 [ 10192 ] {c896d58d-9485-464b-91cb-6dd7d1ad8585} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0): While executing S3Source. (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53494) (comment: 01801_s3_cluster_count.sql) (in query: SELECT count() AS `COUNT()` FROM s3Cluster('test_cluster_two_shards_localhost', 'http://localhost:11111/test/{a,b,c}.tsv', 'TSV', '`c1` Nullable(Int64), `c2` Nullable(Int64), `c3` Nullable(Int64)') AS __table1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000267f6ee2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000267f68f4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000267daa16 9. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSource.cpp:0: DB::StorageObjectStorageSource::generate() @ 0x00000000267db7f6 10. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 11. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 12. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 13. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 14. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 15. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 16. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:46.651022 [ 2823 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:46.656547 [ 10192 ] {c896d58d-9485-464b-91cb-6dd7d1ad8585} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0): While executing S3Source. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000267f6ee2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000267f68f4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000267daa16 9. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSource.cpp:0: DB::StorageObjectStorageSource::generate() @ 0x00000000267db7f6 10. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 11. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 12. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 13. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 14. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 15. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 16. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:46.662462 [ 2843 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:46.736258 [ 13112 ] {d31abdec-ed12-4b48-93c4-b81bf24ad434} executeQuery: Code: 439. DB::Exception: Received from localhost:9000. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0): While executing S3Source. Stack trace: 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000267f6ee2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000267f68f4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000267daa16 9. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSource.cpp:0: DB::StorageObjectStorageSource::generate() @ 0x00000000267db7f6 10. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 11. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 12. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 13. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 14. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 15. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 16. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 : While executing Remote. (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53482) (comment: 01801_s3_cluster_count.sql) (in query: select COUNT() from s3Cluster('test_cluster_two_shards_localhost', 'http://localhost:11111/test/{a,b,c}.tsv');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001aeb974e 3. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000002cee6c6a 4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x000000002cef780e 5. ./build_docker/./src/Client/MultiplexedConnections.cpp:340: DB::MultiplexedConnections::receivePacketUnlocked(std::function) @ 0x000000002cf8e09e 6. ./build_docker/./src/QueryPipeline/RemoteQueryExecutorReadContext.cpp:0: DB::RemoteQueryExecutorReadContext::Task::run(std::function, std::function) @ 0x00000000278ad616 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278ab436 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278aaa44 9. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func>>(unsigned int, unsigned int) @ 0x00000000278a8394 2025.09.25 21:24:46.741663 [ 13112 ] {d31abdec-ed12-4b48-93c4-b81bf24ad434} TCPHandler: Code: 439. DB::Exception: Received from localhost:9000. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0): While executing S3Source. Stack trace: 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000267f6ee2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000267f68f4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000267daa16 9. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorageSource.cpp:0: DB::StorageObjectStorageSource::generate() @ 0x00000000267db7f6 10. ./src/Processors/Chunk.h:110: DB::ISource::tryGenerate() @ 0x000000002d329c18 11. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 12. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 13. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 14. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 15. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 16. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 : While executing Remote. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001aeb974e 3. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000002cee6c6a 4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x000000002cef780e 5. ./build_docker/./src/Client/MultiplexedConnections.cpp:340: DB::MultiplexedConnections::receivePacketUnlocked(std::function) @ 0x000000002cf8e09e 6. ./build_docker/./src/QueryPipeline/RemoteQueryExecutorReadContext.cpp:0: DB::RemoteQueryExecutorReadContext::Task::run(std::function, std::function) @ 0x00000000278ad616 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278ab436 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278aaa44 9. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func>>(unsigned int, unsigned int) @ 0x00000000278a8394 2025.09.25 21:24:46.934153 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:47.134546 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:47.178942 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:47.191769 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:47.482784 [ 13112 ] {5fe4fb6b-d267-4cae-b5bf-21e0f93ddf69} executeQuery: Code: 42. DB::Exception: Number of arguments for function extractAllGroupsHorizontal doesn't match: passed 0, should be 2: In scope SELECT extractAllGroupsHorizontal(). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55434) (comment: 01246_extractAllGroupsHorizontal.sql) (in query: -- error cases SELECT extractAllGroupsHorizontal();), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, unsigned long&, unsigned long&) @ 0x000000000ccd84e0 4. DB::IFunctionOverloadResolver::checkNumberOfArguments(unsigned long) const @ 0x000000000ccd2a84 5. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd35cf 6. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 7. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 8. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 18. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 19. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 21. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 23. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 27. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007f12cd75eac3 30. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:47.484443 [ 13112 ] {5fe4fb6b-d267-4cae-b5bf-21e0f93ddf69} TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractAllGroupsHorizontal doesn't match: passed 0, should be 2: In scope SELECT extractAllGroupsHorizontal(). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, unsigned long&, unsigned long&) @ 0x000000000ccd84e0 4. DB::IFunctionOverloadResolver::checkNumberOfArguments(unsigned long) const @ 0x000000000ccd2a84 5. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd35cf 6. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 7. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 8. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 18. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 19. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 21. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 23. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 27. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007f12cd75eac3 30. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:47.516796 [ 2858 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:47.575967 [ 13112 ] {48b362be-f782-4e93-9a66-345c292f7de3} executeQuery: Code: 42. DB::Exception: Number of arguments for function extractAllGroupsHorizontal doesn't match: passed 1, should be 2: In scope SELECT extractAllGroupsHorizontal('hello'). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55434) (comment: 01246_extractAllGroupsHorizontal.sql) (in query: SELECT extractAllGroupsHorizontal('hello');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, unsigned long&, unsigned long&) @ 0x000000000ccd84e0 4. DB::IFunctionOverloadResolver::checkNumberOfArguments(unsigned long) const @ 0x000000000ccd2a84 5. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd35cf 6. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 7. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 8. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 18. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 19. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 21. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 23. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 27. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007f12cd75eac3 30. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:47.578641 [ 13112 ] {48b362be-f782-4e93-9a66-345c292f7de3} TCPHandler: Code: 42. DB::Exception: Number of arguments for function extractAllGroupsHorizontal doesn't match: passed 1, should be 2: In scope SELECT extractAllGroupsHorizontal('hello'). (NUMBER_OF_ARGUMENTS_DOESNT_MATCH), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, unsigned long&, unsigned long&) @ 0x000000000ccd84e0 4. DB::IFunctionOverloadResolver::checkNumberOfArguments(unsigned long) const @ 0x000000000ccd2a84 5. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd35cf 6. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 7. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 8. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 15. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 16. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 18. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 19. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 21. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 22. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 23. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 24. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 26. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 27. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007f12cd75eac3 30. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:47.653429 [ 2863 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:47.665315 [ 2855 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:47.890421 [ 13112 ] {ef2c46ef-eab0-40ec-b699-9ec1e8796737} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'needle' to function 'extractAllGroupsHorizontal'. Expected: const String or const FixedString, got: UInt8: In scope SELECT extractAllGroupsHorizontal('hello', 123). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55434) (comment: 01246_extractAllGroupsHorizontal.sql) (in query: SELECT extractAllGroupsHorizontal('hello', 123);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception> const&, String, std::basic_string_view> const&, String>(int, FormatStringHelperImpl::type, std::type_identity> const&>::type, std::type_identity::type, std::type_identity> const&>::type, std::type_identity::type>, String&&, std::basic_string_view> const&, String&&, std::basic_string_view> const&, String&&) @ 0x000000000aaec6f5 4. DB::(anonymous namespace)::validateArgumentsImpl(DB::IFunction const&, std::vector> const&, unsigned long, std::vector> const&) @ 0x000000000aae6b1a 5. DB::validateFunctionArguments(DB::IFunction const&, std::vector> const&, std::vector> const&, std::vector> const&) @ 0x000000000aae5e2a 6. DB::FunctionExtractAllGroups<(anonymous namespace)::HorizontalImpl>::getReturnTypeImpl(std::vector> const&) const @ 0x000000000fcbcac9 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:24:47.894802 [ 13112 ] {ef2c46ef-eab0-40ec-b699-9ec1e8796737} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 2nd argument 'needle' to function 'extractAllGroupsHorizontal'. Expected: const String or const FixedString, got: UInt8: In scope SELECT extractAllGroupsHorizontal('hello', 123). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception> const&, String, std::basic_string_view> const&, String>(int, FormatStringHelperImpl::type, std::type_identity> const&>::type, std::type_identity::type, std::type_identity> const&>::type, std::type_identity::type>, String&&, std::basic_string_view> const&, String&&, std::basic_string_view> const&, String&&) @ 0x000000000aaec6f5 4. DB::(anonymous namespace)::validateArgumentsImpl(DB::IFunction const&, std::vector> const&, unsigned long, std::vector> const&) @ 0x000000000aae6b1a 5. DB::validateFunctionArguments(DB::IFunction const&, std::vector> const&, std::vector> const&, std::vector> const&) @ 0x000000000aae5e2a 6. DB::FunctionExtractAllGroups<(anonymous namespace)::HorizontalImpl>::getReturnTypeImpl(std::vector> const&) const @ 0x000000000fcbcac9 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:24:47.906862 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:47.927192 [ 13112 ] {24393bd4-19cb-4752-83e1-10b0eef18946} executeQuery: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'extractAllGroupsHorizontal'. Expected: const String or const FixedString, got: UInt8: In scope SELECT extractAllGroupsHorizontal(123, 'world'). (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55434) (comment: 01246_extractAllGroupsHorizontal.sql) (in query: SELECT extractAllGroupsHorizontal(123, 'world');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception> const&, String, std::basic_string_view> const&, String>(int, FormatStringHelperImpl::type, std::type_identity> const&>::type, std::type_identity::type, std::type_identity> const&>::type, std::type_identity::type>, String&&, std::basic_string_view> const&, String&&, std::basic_string_view> const&, String&&) @ 0x000000000aaec6f5 4. DB::(anonymous namespace)::validateArgumentsImpl(DB::IFunction const&, std::vector> const&, unsigned long, std::vector> const&) @ 0x000000000aae6b1a 5. DB::validateFunctionArguments(DB::IFunction const&, std::vector> const&, std::vector> const&, std::vector> const&) @ 0x000000000aae5e2a 6. DB::FunctionExtractAllGroups<(anonymous namespace)::HorizontalImpl>::getReturnTypeImpl(std::vector> const&) const @ 0x000000000fcbcac9 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:24:47.929389 [ 13112 ] {24393bd4-19cb-4752-83e1-10b0eef18946} TCPHandler: Code: 43. DB::Exception: A value of illegal type was provided as 1st argument 'haystack' to function 'extractAllGroupsHorizontal'. Expected: const String or const FixedString, got: UInt8: In scope SELECT extractAllGroupsHorizontal(123, 'world'). (ILLEGAL_TYPE_OF_ARGUMENT), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception> const&, String, std::basic_string_view> const&, String>(int, FormatStringHelperImpl::type, std::type_identity> const&>::type, std::type_identity::type, std::type_identity> const&>::type, std::type_identity::type>, String&&, std::basic_string_view> const&, String&&, std::basic_string_view> const&, String&&) @ 0x000000000aaec6f5 4. DB::(anonymous namespace)::validateArgumentsImpl(DB::IFunction const&, std::vector> const&, unsigned long, std::vector> const&) @ 0x000000000aae6b1a 5. DB::validateFunctionArguments(DB::IFunction const&, std::vector> const&, std::vector> const&, std::vector> const&) @ 0x000000000aae5e2a 6. DB::FunctionExtractAllGroups<(anonymous namespace)::HorizontalImpl>::getReturnTypeImpl(std::vector> const&) const @ 0x000000000fcbcac9 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:24:47.954929 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:47.976473 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:48.098721 [ 13112 ] {8ba7dded-8437-4241-a725-a3f26ac2996c} executeQuery: Code: 427. DB::Exception: OptimizedRegularExpression: cannot compile re2: (((, error: missing ): (((. Look at https://github.com/google/re2/wiki/Syntax for reference. Please note that if you specify regex as an SQL string literal, the slashes have to be additionally escaped. For example, to match an opening brace, write '\(' -- the first slash is for SQL and the second one is for regex: In scope SELECT extractAllGroupsHorizontal('hello world', '((('). (CANNOT_COMPILE_REGEXP) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55434) (comment: 01246_extractAllGroupsHorizontal.sql) (in query: SELECT extractAllGroupsHorizontal('hello world', '(((');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String const&) @ 0x000000000aab393a 4. ./build_docker/./src/Common/OptimizedRegularExpression.cpp:508: OptimizedRegularExpression::OptimizedRegularExpression(String const&, int) @ 0x000000001ae083d2 5. DB::FunctionExtractAllGroups<(anonymous namespace)::HorizontalImpl>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000fcb9d7b 6. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 7. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 9. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 10. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 11. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 12. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 23. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 28. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 2025.09.25 21:24:48.101089 [ 13112 ] {8ba7dded-8437-4241-a725-a3f26ac2996c} TCPHandler: Code: 427. DB::Exception: OptimizedRegularExpression: cannot compile re2: (((, error: missing ): (((. Look at https://github.com/google/re2/wiki/Syntax for reference. Please note that if you specify regex as an SQL string literal, the slashes have to be additionally escaped. For example, to match an opening brace, write '\(' -- the first slash is for SQL and the second one is for regex: In scope SELECT extractAllGroupsHorizontal('hello world', '((('). (CANNOT_COMPILE_REGEXP), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String const&) @ 0x000000000aab393a 4. ./build_docker/./src/Common/OptimizedRegularExpression.cpp:508: OptimizedRegularExpression::OptimizedRegularExpression(String const&, int) @ 0x000000001ae083d2 5. DB::FunctionExtractAllGroups<(anonymous namespace)::HorizontalImpl>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000fcb9d7b 6. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 7. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 9. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 10. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 11. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 12. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 23. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 28. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 2025.09.25 21:24:48.176725 [ 13112 ] {3ab09c00-8b84-4a20-a6c7-a83255429e31} executeQuery: Code: 44. DB::Exception: A value of illegal type was provided as 2nd argument 'needle' to function 'extractAllGroupsHorizontal'. Expected: const String or const FixedString, got: String: In scope SELECT extractAllGroupsHorizontal('hello world', materialize('\\w+')). (ILLEGAL_COLUMN) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55434) (comment: 01246_extractAllGroupsHorizontal.sql) (in query: SELECT extractAllGroupsHorizontal('hello world', materialize('\\w+'));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception> const&, String, std::basic_string_view> const&, String>(int, FormatStringHelperImpl::type, std::type_identity> const&>::type, std::type_identity::type, std::type_identity> const&>::type, std::type_identity::type>, String&&, std::basic_string_view> const&, String&&, std::basic_string_view> const&, String&&) @ 0x000000000aaec6f5 4. DB::(anonymous namespace)::validateArgumentsImpl(DB::IFunction const&, std::vector> const&, unsigned long, std::vector> const&) @ 0x000000000aae6b1a 5. DB::validateFunctionArguments(DB::IFunction const&, std::vector> const&, std::vector> const&, std::vector> const&) @ 0x000000000aae5e2a 6. DB::FunctionExtractAllGroups<(anonymous namespace)::HorizontalImpl>::getReturnTypeImpl(std::vector> const&) const @ 0x000000000fcbcac9 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:24:48.187706 [ 13112 ] {3ab09c00-8b84-4a20-a6c7-a83255429e31} TCPHandler: Code: 44. DB::Exception: A value of illegal type was provided as 2nd argument 'needle' to function 'extractAllGroupsHorizontal'. Expected: const String or const FixedString, got: String: In scope SELECT extractAllGroupsHorizontal('hello world', materialize('\\w+')). (ILLEGAL_COLUMN), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception> const&, String, std::basic_string_view> const&, String>(int, FormatStringHelperImpl::type, std::type_identity> const&>::type, std::type_identity::type, std::type_identity> const&>::type, std::type_identity::type>, String&&, std::basic_string_view> const&, String&&, std::basic_string_view> const&, String&&) @ 0x000000000aaec6f5 4. DB::(anonymous namespace)::validateArgumentsImpl(DB::IFunction const&, std::vector> const&, unsigned long, std::vector> const&) @ 0x000000000aae6b1a 5. DB::validateFunctionArguments(DB::IFunction const&, std::vector> const&, std::vector> const&, std::vector> const&) @ 0x000000000aae5e2a 6. DB::FunctionExtractAllGroups<(anonymous namespace)::HorizontalImpl>::getReturnTypeImpl(std::vector> const&) const @ 0x000000000fcbcac9 7. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::vector> const&) const @ 0x000000000ccd376e 8. DB::IFunctionOverloadResolver::getReturnType(std::vector> const&) const @ 0x000000000ccd3176 9. DB::IFunctionOverloadResolver::build(std::vector> const&) const @ 0x000000000ccd4343 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x0000000028399ff0 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 17. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 19. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 20. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 21. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 23. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 24. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 25. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 26. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 28. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 29. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 2025.09.25 21:24:48.379685 [ 13112 ] {8f04a36d-cf3d-4a08-b907-0773c0445319} executeQuery: Code: 36. DB::Exception: There are no groups in regexp: \w+: In scope SELECT extractAllGroupsHorizontal('hello world', '\\w+'). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55434) (comment: 01246_extractAllGroupsHorizontal.sql) (in query: SELECT extractAllGroupsHorizontal('hello world', '\\w+');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. DB::FunctionExtractAllGroups<(anonymous namespace)::HorizontalImpl>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000fcbbbf3 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:24:48.391392 [ 13112 ] {8f04a36d-cf3d-4a08-b907-0773c0445319} TCPHandler: Code: 36. DB::Exception: There are no groups in regexp: \w+: In scope SELECT extractAllGroupsHorizontal('hello world', '\\w+'). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. DB::FunctionExtractAllGroups<(anonymous namespace)::HorizontalImpl>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000fcbbbf3 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:24:48.430611 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:48.507559 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:48.530939 [ 2889 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:48.534975 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:48.602890 [ 13112 ] {d760b7fd-c3a1-4072-a192-c45b8c006a3e} executeQuery: Code: 128. DB::Exception: Too many matches per row (> 0) in the result of function extractAllGroupsHorizontal: In scope SELECT extractAllGroupsHorizontal('hello world', '(\\w+)') SETTINGS regexp_max_matches_per_row = 0. (TOO_LARGE_ARRAY_SIZE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55434) (comment: 01246_extractAllGroupsHorizontal.sql) (in query: SELECT extractAllGroupsHorizontal('hello world', '(\\w+)') SETTINGS regexp_max_matches_per_row = 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception const&, String>(int, FormatStringHelperImpl const&>::type, std::type_identity::type>, DB::SettingFieldNumber const&, String&&) @ 0x000000000fcbd3ba 4. DB::FunctionExtractAllGroups<(anonymous namespace)::HorizontalImpl>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000fcbb7b6 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:24:48.614535 [ 13112 ] {d760b7fd-c3a1-4072-a192-c45b8c006a3e} TCPHandler: Code: 128. DB::Exception: Too many matches per row (> 0) in the result of function extractAllGroupsHorizontal: In scope SELECT extractAllGroupsHorizontal('hello world', '(\\w+)') SETTINGS regexp_max_matches_per_row = 0. (TOO_LARGE_ARRAY_SIZE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception const&, String>(int, FormatStringHelperImpl const&>::type, std::type_identity::type>, DB::SettingFieldNumber const&, String&&) @ 0x000000000fcbd3ba 4. DB::FunctionExtractAllGroups<(anonymous namespace)::HorizontalImpl>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000fcbb7b6 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:24:48.655520 [ 2892 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:48.668761 [ 2896 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:48.687252 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:48.692782 [ 13112 ] {a40b90c8-9bf7-440e-9c87-3f233c4c934a} executeQuery: Code: 128. DB::Exception: Too many matches per row (> 1) in the result of function extractAllGroupsHorizontal: In scope SELECT extractAllGroupsHorizontal('hello world', '(\\w+)') SETTINGS regexp_max_matches_per_row = 1. (TOO_LARGE_ARRAY_SIZE) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55434) (comment: 01246_extractAllGroupsHorizontal.sql) (in query: SELECT extractAllGroupsHorizontal('hello world', '(\\w+)') SETTINGS regexp_max_matches_per_row = 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception const&, String>(int, FormatStringHelperImpl const&>::type, std::type_identity::type>, DB::SettingFieldNumber const&, String&&) @ 0x000000000fcbd3ba 4. DB::FunctionExtractAllGroups<(anonymous namespace)::HorizontalImpl>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000fcbb7b6 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:24:48.697517 [ 13112 ] {a40b90c8-9bf7-440e-9c87-3f233c4c934a} TCPHandler: Code: 128. DB::Exception: Too many matches per row (> 1) in the result of function extractAllGroupsHorizontal: In scope SELECT extractAllGroupsHorizontal('hello world', '(\\w+)') SETTINGS regexp_max_matches_per_row = 1. (TOO_LARGE_ARRAY_SIZE), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception const&, String>(int, FormatStringHelperImpl const&>::type, std::type_identity::type>, DB::SettingFieldNumber const&, String&&) @ 0x000000000fcbd3ba 4. DB::FunctionExtractAllGroups<(anonymous namespace)::HorizontalImpl>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000fcbb7b6 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:24:49.224356 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:49.288801 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:49.304142 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:49.533683 [ 2900 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:49.657526 [ 2907 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:49.671114 [ 2913 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:49.800096 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:49.944833 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:49.972551 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:50.039527 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:50.537662 [ 2934 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:50.660201 [ 2940 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:50.683296 [ 2935 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:50.891677 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:51.141970 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:51.344669 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:51.376572 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:51.633776 [ 2956 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:51.662384 [ 2957 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:51.670161 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:51.690765 [ 2960 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:51.834718 [ 13112 ] {4dfaf990-b4ae-4f63-b3ac-e201edd6e9dd} executeQuery: Code: 452. DB::Exception: Setting allow_experimental_object_type should not be changed. (SETTING_CONSTRAINT_VIOLATION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55442) (comment: 01825_type_json_empty_string.sql) (in query: SET allow_experimental_object_type = 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges const&, DB::SettingSource) const @ 0x0000000026f807e3 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraints(DB::SettingsChanges const&, DB::SettingSource) @ 0x000000002853b6ad 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::execute() @ 0x00000000294f603b 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:51.842960 [ 13112 ] {4dfaf990-b4ae-4f63-b3ac-e201edd6e9dd} TCPHandler: Code: 452. DB::Exception: Setting allow_experimental_object_type should not be changed. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges const&, DB::SettingSource) const @ 0x0000000026f807e3 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraints(DB::SettingsChanges const&, DB::SettingSource) @ 0x000000002853b6ad 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::execute() @ 0x00000000294f603b 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:51.950387 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:51.981309 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:52.405516 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:52.407607 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:52.427836 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:52.635966 [ 2991 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:52.664425 [ 2982 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:52.693587 [ 2994 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:52.821819 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:53.073205 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:53.303328 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:53.309931 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:53.638662 [ 3016 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:53.667034 [ 3021 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:53.700115 [ 3029 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:53.875767 [ 13110 ] {6ace8fb9-e8a2-47d3-98d7-f13ec0e46c7e} executeQuery: Code: 48. DB::Exception: Truncate is not supported by storage Null. (NOT_IMPLEMENTED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37672) (comment: 02231_buffer_aggregate_states_leak.sql) (in query: drop table in_02231;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:53.882659 [ 13110 ] {6ace8fb9-e8a2-47d3-98d7-f13ec0e46c7e} TCPHandler: Code: 48. DB::Exception: Truncate is not supported by storage Null. (NOT_IMPLEMENTED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::IStorage::truncate(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr, DB::TableExclusiveLockHolder&) @ 0x00000000260e9270 4. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:279: DB::InterpreterDropQuery::executeToTableImpl(std::shared_ptr const&, DB::ASTDropQuery&, std::shared_ptr&, StrongTypedef, DB::UUIDTag>&) @ 0x00000000291f066f 5. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeToTable(DB::ASTDropQuery&) @ 0x00000000291ed665 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebd68 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 11. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:53.901064 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:54.254734 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:54.486973 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:54.603629 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:54.641067 [ 3056 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:54.669432 [ 3054 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:54.702692 [ 3058 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:54.783319 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:54.816973 [ 13112 ] {a7002f1b-e4b6-40a9-8cd1-84cbd221d2e6} executeQuery: Code: 57. DB::Exception: Table test_5.test already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from 0.0.0.0:0) (comment: 03214_json_typed_dynamic_path.sql) (in query: /* ddl_entry=query-0000000086 */ CREATE TABLE test_5.test UUID 'a1cfd11f-8d2e-4338-91e7-9b54b794bedb' (`json` JSON(a Dynamic)) ENGINE = MergeTree ORDER BY tuple() SETTINGS min_rows_for_wide_part = 10000000, min_bytes_for_wide_part = 10000000, index_granularity = 64501, ratio_of_defaults_for_sparse_serialization = 1., replace_long_file_name_to_hash = true, max_file_name_length = 128, min_bytes_for_full_part_storage = 536870912, compact_parts_max_bytes_to_buffer = 33743752, compact_parts_max_granules_to_buffer = 213, compact_parts_merge_max_bytes_to_prefetch_part = 25304433, merge_max_block_size = 3332, old_parts_lifetime = 391., prefer_fetch_merged_part_size_threshold = 1, vertical_merge_algorithm_min_rows_to_activate = 1000000, vertical_merge_algorithm_min_columns_to_activate = 1, min_merge_bytes_to_use_direct_io = 7902803964, index_granularity_bytes = 23403725, concurrent_part_removal_threshold = 10, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = false, marks_compress_block_size = 75351, primary_key_compress_block_size = 92191), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:54.850752 [ 13112 ] {a7002f1b-e4b6-40a9-8cd1-84cbd221d2e6} executeQuery: Code: 57. DB::Exception: Table test_5.test already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55456) (comment: 03214_json_typed_dynamic_path.sql) (in query: CREATE TABLE test (`json` JSON(a Dynamic)) ENGINE = MergeTree ORDER BY tuple() SETTINGS min_rows_for_wide_part = 10000000, min_bytes_for_wide_part = 10000000, index_granularity = 64501, ratio_of_defaults_for_sparse_serialization = 1., replace_long_file_name_to_hash = true, max_file_name_length = 128, min_bytes_for_full_part_storage = 536870912, compact_parts_max_bytes_to_buffer = 33743752, compact_parts_max_granules_to_buffer = 213, compact_parts_merge_max_bytes_to_prefetch_part = 25304433, merge_max_block_size = 3332, old_parts_lifetime = 391., prefer_fetch_merged_part_size_threshold = 1, vertical_merge_algorithm_min_rows_to_activate = 1000000, vertical_merge_algorithm_min_columns_to_activate = 1, min_merge_bytes_to_use_direct_io = 7902803964, index_granularity_bytes = 23403725, concurrent_part_removal_threshold = 10, allow_vertical_merges_from_compact_to_wide_parts = true, cache_populated_by_fetch = false, marks_compress_block_size = 75351, primary_key_compress_block_size = 92191), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:54.856343 [ 13112 ] {a7002f1b-e4b6-40a9-8cd1-84cbd221d2e6} TCPHandler: Code: 57. DB::Exception: Table test_5.test already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 9. ./build_docker/./src/Interpreters/DDLWorker.cpp:498: DB::DDLWorker::tryExecuteQuery(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028686d70 10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DDLWorker::processTask(DB::DDLTaskBase&, std::shared_ptr const&) @ 0x0000000028682a53 11. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:331: DB::DatabaseReplicatedDDLWorker::tryEnqueueAndExecuteEntry(DB::DDLLogEntry&, std::shared_ptr) @ 0x0000000027dab022 12. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: DB::DatabaseReplicated::tryEnqueueReplicatedDDL(std::shared_ptr const&, std::shared_ptr, DB::QueryFlags) @ 0x0000000027d5fb0b 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac6af 14. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:54.979324 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:55.030571 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:55.396832 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:55.501152 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:55.642860 [ 3089 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:55.671810 [ 3079 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:55.705475 [ 3086 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:55.751232 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:55.779456 [ 13112 ] {ae004cc4-361d-43f3-ad8d-d2f860e823d3} executeQuery: Code: 703. DB::Exception: Expected identifier 't.t.t' to contain 1 or 2 parts to be resolved as table expression. In scope system.tables. (INVALID_IDENTIFIER) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55476) (comment: 02963_invalid_identifier.sql) (in query: SELECT t.t.t.* FROM system.tables WHERE database = currentDatabase();), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/IdentifierResolver.cpp:900: DB::IdentifierResolver::tryResolveIdentifierFromTableExpression(DB::IdentifierLookup const&, std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x0000000028435e59 5. ./build_docker/./src/Analyzer/Resolve/IdentifierResolver.cpp:1493: DB::IdentifierResolver::tryResolveIdentifierFromJoinTreeNode(DB::IdentifierLookup const&, std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x00000000284392e1 6. ./build_docker/./src/Analyzer/Resolve/IdentifierResolver.cpp:1526: DB::IdentifierResolver::tryResolveIdentifierFromJoinTree(DB::IdentifierLookup const&, DB::IdentifierResolveScope&) @ 0x000000002843cf94 7. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::tryResolveIdentifier(DB::IdentifierLookup const&, DB::IdentifierResolveScope&, DB::IdentifierResolveSettings) @ 0x00000000283848ca 8. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQualifiedMatcher(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002838d9cc 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveMatcher(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283a74cb 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367efa 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:55.783282 [ 13112 ] {ae004cc4-361d-43f3-ad8d-d2f860e823d3} TCPHandler: Code: 703. DB::Exception: Expected identifier 't.t.t' to contain 1 or 2 parts to be resolved as table expression. In scope system.tables. (INVALID_IDENTIFIER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Analyzer/Resolve/IdentifierResolver.cpp:900: DB::IdentifierResolver::tryResolveIdentifierFromTableExpression(DB::IdentifierLookup const&, std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x0000000028435e59 5. ./build_docker/./src/Analyzer/Resolve/IdentifierResolver.cpp:1493: DB::IdentifierResolver::tryResolveIdentifierFromJoinTreeNode(DB::IdentifierLookup const&, std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x00000000284392e1 6. ./build_docker/./src/Analyzer/Resolve/IdentifierResolver.cpp:1526: DB::IdentifierResolver::tryResolveIdentifierFromJoinTree(DB::IdentifierLookup const&, DB::IdentifierResolveScope&) @ 0x000000002843cf94 7. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::QueryAnalyzer::tryResolveIdentifier(DB::IdentifierLookup const&, DB::IdentifierResolveScope&, DB::IdentifierResolveSettings) @ 0x00000000283848ca 8. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQualifiedMatcher(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002838d9cc 9. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveMatcher(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283a74cb 10. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367efa 11. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 12. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 16. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 18. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 19. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 20. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 22. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 23. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 24. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 25. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 27. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 28. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 29. asan_thread_start(void*) @ 0x000000000aa43059 30. ? @ 0x00007f12cd75eac3 31. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:56.299871 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:56.426060 [ 13110 ] {3b8a751b-7d97-4022-802d-0262b1c63a42} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:57380) (comment: 00748_insert_array_with_null.sql) (in query: DROP DATABASE IF EXISTS test_gc66y9g2 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:56.430234 [ 13110 ] {3b8a751b-7d97-4022-802d-0262b1c63a42} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:56.610948 [ 13112 ] {78ea7587-9e70-4d79-a8c7-f41ac3ff771c} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:57390) (comment: 00748_insert_array_with_null.sql) (in query: DROP DATABASE IF EXISTS test_gc66y9g2 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:24:56.620360 [ 13112 ] {78ea7587-9e70-4d79-a8c7-f41ac3ff771c} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:56.646218 [ 3113 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:56.675050 [ 3109 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:56.714000 [ 3110 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:56.749871 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:56.771606 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:57.377851 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:57.564628 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:57.649546 [ 3138 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:57.649855 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:57.678257 [ 3139 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:57.716805 [ 3136 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:58.495542 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:58.652885 [ 3153 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:58.667156 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:58.719688 [ 3161 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:58.728099 [ 3171 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:58.768803 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:58.798549 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:59.325316 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:59.655879 [ 3201 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:59.722424 [ 3192 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:59.730932 [ 3169 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:59.748880 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:59.769766 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:24:59.825155 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:00.658058 [ 3222 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:00.665556 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:00.794982 [ 3244 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:00.795546 [ 3231 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:00.975225 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:00.993237 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:01.540793 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:01.645580 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:01.660830 [ 3255 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:01.688461 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:01.797746 [ 3254 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:01.798447 [ 2756 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:02.391108 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:02.496181 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:02.585808 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:02.663758 [ 2775 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:02.800991 [ 2753 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:02.802080 [ 2778 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:03.128821 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:03.378735 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:03.379236 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:03.670463 [ 2800 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:03.804408 [ 2797 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:03.805179 [ 2791 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:03.990795 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:04.245055 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:04.272817 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:04.673639 [ 2814 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:04.807389 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:04.807702 [ 2819 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:04.808106 [ 2830 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:04.869208 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:05.141275 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:05.676832 [ 2844 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:05.811177 [ 2840 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:05.811878 [ 2841 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:05.896190 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:06.139818 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:06.211872 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:06.672987 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:06.679081 [ 2873 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:06.814099 [ 2875 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:06.815025 [ 2877 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:06.970984 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:07.013002 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:07.561070 [ 3365 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {02260661-95d0-4eca-a73d-2543d99ce5b9::202509_276_306_6}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:07.596253 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:07.681750 [ 2898 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:07.818831 [ 2906 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:07.819307 [ 2904 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:08.142321 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:08.166529 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:08.685145 [ 2919 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:08.742246 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:08.821417 [ 2917 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:08.821671 [ 2914 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:09.044450 [ 13129 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:09.410370 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:09.449893 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:09.534506 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:09.693081 [ 2946 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:09.825076 [ 2947 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:09.825548 [ 2945 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:10.353243 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:10.513924 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:10.554900 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:10.695615 [ 2960 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:10.827958 [ 2973 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:10.828299 [ 2967 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:10.971825 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:11.003859 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:11.083071 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:11.698037 [ 2981 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:11.729208 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:11.822224 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:11.831783 [ 3005 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:11.831903 [ 2995 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:11.937500 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:12.536288 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:12.566482 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:12.700333 [ 3020 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:12.777788 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:12.834135 [ 3024 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:12.835080 [ 3023 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:13.034097 [ 13129 ] {d58fc89e-6822-43dd-aca9-30a16ba80fab} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:57876) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:13.037454 [ 13129 ] {d58fc89e-6822-43dd-aca9-30a16ba80fab} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:13.450274 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:13.617052 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:13.621892 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:13.629361 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:13.709269 [ 3049 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:13.836236 [ 3053 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:13.837102 [ 3050 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:14.198835 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:14.401482 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:14.426510 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:14.711350 [ 3088 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:14.839633 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:14.859790 [ 3090 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:14.860204 [ 3085 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:14.965148 [ 13130 ] {276c3ece-779a-46ae-af5e-1d15321aa89e} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:57910) (comment: 03008_local_plain_rewritable.sh) (in query: select (*) from 03008_test_local_mt order by tuple(a, b) limit 10;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:14.969633 [ 13130 ] {276c3ece-779a-46ae-af5e-1d15321aa89e} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:14.996057 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:15.016645 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:15.093891 [ 13129 ] {6cb30e48-b995-4481-9dd0-be6710b98bd8} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:57908) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:15.097706 [ 13129 ] {6cb30e48-b995-4481-9dd0-be6710b98bd8} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:15.237292 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:15.713835 [ 3110 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:15.862247 [ 3112 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:15.862800 [ 3115 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:16.023297 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:16.210246 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:16.223222 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:16.441660 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:16.803989 [ 3150 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:16.864994 [ 3145 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:16.866033 [ 3147 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:17.491915 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:17.538567 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:17.561695 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:17.649292 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:17.806329 [ 3175 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:17.867806 [ 3185 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:17.868291 [ 3184 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:18.393328 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:18.627211 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:18.728786 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:18.748313 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:18.808613 [ 3207 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:18.869792 [ 3179 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:18.870276 [ 3169 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:19.161706 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:19.182605 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:19.511365 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:19.702253 [ 13130 ] {fad379c5-6d32-4aff-87b0-13114362feb8} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:41704) (comment: 03008_local_plain_rewritable.sh) (in query: select (*) from 03008_test_local_mt order by tuple(a, b) limit 10;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:19.708420 [ 13130 ] {fad379c5-6d32-4aff-87b0-13114362feb8} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::getAnalysisResult() const @ 0x000000002de41345 10. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:2022: DB::ReadFromMergeTree::initializePipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&) @ 0x000000002de4404c 11. ./build_docker/./src/Processors/QueryPlan/ISourceStep.cpp:24: DB::ISourceStep::updatePipeline(std::vector>, std::allocator>>>, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddb933c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddff11e 13. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:19.811370 [ 3220 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:19.872419 [ 3215 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:19.873165 [ 3226 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:20.118212 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:20.162932 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:20.303040 [ 13129 ] {d3220b70-c517-4765-945f-b0e688ef9693} executeQuery: Code: 344. DB::Exception: Mutations are not supported for immutable disk '03008_local_plain_rewritable'. (SUPPORT_IS_DISABLED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:41718) (comment: 03008_local_plain_rewritable.sh) (in query: alter table 03008_test_local_mt update c = 0 where a % 2 = 1;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:3636: DB::MergeTreeData::checkMutationIsPossible(DB::MutationCommands const&, DB::Settings const&) const @ 0x000000002c4a380c 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000029174782 6. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:20.318171 [ 13129 ] {d3220b70-c517-4765-945f-b0e688ef9693} TCPHandler: Code: 344. DB::Exception: Mutations are not supported for immutable disk '03008_local_plain_rewritable'. (SUPPORT_IS_DISABLED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:3636: DB::MergeTreeData::checkMutationIsPossible(DB::MutationCommands const&, DB::Settings const&) const @ 0x000000002c4a380c 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000029174782 6. ./build_docker/./src/Interpreters/InterpreterAlterQuery.cpp:74: DB::InterpreterAlterQuery::execute() @ 0x0000000029170711 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:20.572621 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:20.707522 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:20.726807 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:20.814318 [ 3248 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:20.875427 [ 3245 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:20.876306 [ 3252 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:21.004560 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:21.290193 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:21.365603 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:21.820324 [ 2757 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:21.880378 [ 2775 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:21.881252 [ 2769 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:21.910084 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:21.927050 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:21.959132 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:22.263957 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:22.823367 [ 2798 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:22.839565 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:22.883376 [ 2803 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:22.885389 [ 2804 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:22.891722 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:23.096326 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:23.251978 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:23.280349 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:23.645425 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:23.691518 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:23.830778 [ 2832 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:23.973653 [ 2835 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:23.974360 [ 2823 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:24.111836 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:24.726460 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:24.785791 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:24.820086 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:24.833718 [ 2864 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:24.976609 [ 2863 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:24.977458 [ 2855 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:25.008548 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:25.411018 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:25.429596 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:25.666009 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:25.836227 [ 2874 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:25.980242 [ 2890 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:25.980425 [ 2883 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:26.250932 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:26.282070 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:26.387604 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:26.838022 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:26.848526 [ 2903 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:26.899323 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:26.927338 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:26.983912 [ 2915 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:26.985513 [ 2907 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:27.405111 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:27.422509 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:27.466504 [ 13112 ] {c277f89e-154b-46fc-b237-c9f8dcb6c6bf} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50354) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:25:27.470531 [ 13112 ] {c277f89e-154b-46fc-b237-c9f8dcb6c6bf} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:25:27.834654 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:27.851431 [ 2934 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:27.988754 [ 2929 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:28.079426 [ 2940 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:28.158072 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:28.222627 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:28.222492 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:28.855220 [ 2951 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:28.991881 [ 2955 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:29.174818 [ 2964 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:29.292348 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:29.368393 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:29.409232 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:29.608601 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:29.788911 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:29.858678 [ 2991 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:29.917337 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:29.995194 [ 2994 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:30.176688 [ 3000 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:30.205989 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:30.446464 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:30.471200 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:30.688574 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:30.863128 [ 3025 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:31.004390 [ 3019 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:31.178408 [ 3023 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:31.454224 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:31.542907 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:31.579902 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:31.866064 [ 3053 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:32.007632 [ 3059 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:32.189374 [ 3064 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:32.198067 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:32.229552 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:32.396282 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:32.421220 [ 13129 ] {27ccb135-e410-4a2e-b9a5-ef3685a0690e} executeQuery: Code: 57. DB::Exception: Table test_0pzmqtun.test_mv already exists. (TABLE_ALREADY_EXISTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50370) (comment: 02943_create_query_interpreter_sample_block_fix.sql) (in query: CREATE MATERIALIZED VIEW test_mv ENGINE=MergeTree ORDER BY arr AS WITH (SELECT regex FROM regex_test_table) AS constant_value SELECT extractAll(concat(toString(number), 'a'), assumeNotNull(constant_value)) AS arr FROM test_table;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:32.427209 [ 13129 ] {27ccb135-e410-4a2e-b9a5-ef3685a0690e} TCPHandler: Code: 57. DB::Exception: Table test_0pzmqtun.test_mv already exists. (TABLE_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x000000000aa91e9a 4. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1558: DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x00000000291bc5ff 5. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:0: DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x00000000291ac835 6. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c3415 7. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 8. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:32.682960 [ 13130 ] {0f78f2ca-105e-49d2-9690-5b8e1bbcd87e} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50810) (comment: 02943_create_query_interpreter_sample_block_fix.sql) (in query: DROP DATABASE IF EXISTS test_0pzmqtun ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000291f4e6a 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 13. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 14. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 15. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 19. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 20. asan_thread_start(void*) @ 0x000000000aa43059 21. ? @ 0x00007f12cd75eac3 22. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:32.686871 [ 13130 ] {0f78f2ca-105e-49d2-9690-5b8e1bbcd87e} DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000291f4e6a 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 13. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 14. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 15. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 16. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 18. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 19. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 20. asan_thread_start(void*) @ 0x000000000aa43059 21. ? @ 0x00007f12cd75eac3 22. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:32.794234 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:32.876183 [ 3073 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:33.010874 [ 3079 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:33.191895 [ 3090 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:33.397796 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:33.437485 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:33.533073 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:33.700491 [ 13130 ] {54a835e6-16d0-4c68-9e5b-f905041db532} executeQuery: Code: 452. DB::Exception: Setting allow_experimental_analyzer should not be changed. (SETTING_CONSTRAINT_VIOLATION) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50398) (comment: 02478_window_frame_type_groups.sql) (in query: SET enable_analyzer = 0;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges const&, DB::SettingSource) const @ 0x0000000026f807e3 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraints(DB::SettingsChanges const&, DB::SettingSource) @ 0x000000002853b6ad 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::execute() @ 0x00000000294f603b 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:33.704046 [ 13130 ] {54a835e6-16d0-4c68-9e5b-f905041db532} TCPHandler: Code: 452. DB::Exception: Setting allow_experimental_analyzer should not be changed. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h:100: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges const&, DB::SettingSource) const @ 0x0000000026f807e3 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraints(DB::SettingsChanges const&, DB::SettingSource) @ 0x000000002853b6ad 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSetQuery::execute() @ 0x00000000294f603b 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:33.758273 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:33.882338 [ 3122 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:33.979240 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:33.994347 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:34.013565 [ 3108 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:34.193735 [ 3120 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:34.416033 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:34.885549 [ 3130 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:35.015703 [ 3151 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:35.049956 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:35.113012 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:35.196831 [ 3146 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:35.210609 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:35.888096 [ 3173 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:35.889412 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:35.903262 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:36.018451 [ 3175 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:36.112942 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:36.200033 [ 3167 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:36.563218 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:36.585232 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:36.809247 [ 13129 ] {781edced-094f-4c22-8a71-c6f308187526} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=734). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50420) (comment: 01069_materialized_view_alter_target_table.sql) (in query: SELECT * FROM mv_target ORDER BY a;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:36.816250 [ 13129 ] {781edced-094f-4c22-8a71-c6f308187526} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=734). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:36.891078 [ 3179 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:37.021503 [ 3214 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:37.112332 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:37.118811 [ 13112 ] {be24dd14-003c-4321-b00d-d146b9e3a61c} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34648) (comment: 01069_materialized_view_alter_target_table.sql) (in query: DROP DATABASE IF EXISTS test_p538erv3 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:37.123840 [ 13112 ] {be24dd14-003c-4321-b00d-d146b9e3a61c} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:37.146216 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:37.203468 [ 3212 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:37.214642 [ 13129 ] {080dd9bf-d33b-4cab-ab92-9c497cf526de} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34662) (comment: 01069_materialized_view_alter_target_table.sql) (in query: DROP DATABASE IF EXISTS test_p538erv3 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:37.219036 [ 13129 ] {080dd9bf-d33b-4cab-ab92-9c497cf526de} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:37.247610 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:37.304672 [ 13130 ] {86d4d842-959e-4cf1-95aa-25aebb5448b3} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:34664) (comment: 01069_materialized_view_alter_target_table.sql) (in query: DROP DATABASE IF EXISTS test_p538erv3 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:37.309014 [ 13130 ] {86d4d842-959e-4cf1-95aa-25aebb5448b3} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:37.852672 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:37.893265 [ 3246 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:38.024646 [ 3250 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:38.233479 [ 3252 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:38.246433 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:38.348466 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:38.651835 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:38.677457 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:38.774408 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:38.895727 [ 2766 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:39.028140 [ 2779 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:39.320648 [ 2775 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:39.686010 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:39.687364 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:39.777347 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:39.898382 [ 2785 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:40.031263 [ 2794 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:40.323690 [ 2797 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:40.327801 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:40.357276 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:40.650580 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:40.676165 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:40.720193 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:40.901450 [ 2827 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:41.034287 [ 2821 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:41.327121 [ 2825 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:41.607781 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:41.634295 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:41.700485 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:41.905334 [ 2824 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:41.999826 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:42.037546 [ 2853 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:42.100729 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:42.330362 [ 2863 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:42.435874 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:42.641871 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:42.652284 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:42.657670 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:42.781720 [ 10537 ] {72536b3c-f7a6-4b2b-a5a9-9f2b1b4726fa} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53108) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:25:42.785492 [ 10537 ] {72536b3c-f7a6-4b2b-a5a9-9f2b1b4726fa} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:25:42.910717 [ 2875 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:43.040053 [ 2885 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:43.303729 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:43.310280 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:43.333527 [ 2882 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:43.341373 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:43.603288 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:43.913835 [ 2906 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:44.043217 [ 2908 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:44.336089 [ 2912 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:44.397493 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:44.434232 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:44.522150 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:44.917263 [ 2920 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:45.031874 [ 3355 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {f81dbc1c-a1bd-4968-9514-f0c97137169a::202509_1_140_28}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:45.045951 [ 2929 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:45.275166 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:45.338733 [ 2947 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:45.401135 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:45.510930 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:45.920216 [ 2956 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:46.048784 [ 2965 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:46.056302 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:46.340819 [ 2978 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:46.384613 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:46.413316 [ 13129 ] {db50f673-72ee-4f94-8594-2fb7cfe35d23} executeQuery: Code: 36. DB::Exception: JOIN INNER JOIN ... USING (id, id, id) identifier 'id' appears more than once in USING clause. (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53118) (comment: 02382_analyzer_matcher_join_using.sql) (in query: SELECT * FROM test_table_join_1 AS t1 INNER JOIN test_table_join_2 AS t2 USING (id, id, id) ORDER BY id, t1.value;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String const&) @ 0x000000000c30ea7a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5003: DB::QueryAnalyzer::resolveJoin(std::shared_ptr&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x00000000283c086d 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x00000000283ba42e 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ad09 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 8. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 9. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 12. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 13. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 15. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 17. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 21. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 22. asan_thread_start(void*) @ 0x000000000aa43059 23. ? @ 0x00007f12cd75eac3 24. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:46.416559 [ 13129 ] {db50f673-72ee-4f94-8594-2fb7cfe35d23} TCPHandler: Code: 36. DB::Exception: JOIN INNER JOIN ... USING (id, id, id) identifier 'id' appears more than once in USING clause. (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String const&) @ 0x000000000c30ea7a 4. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:5003: DB::QueryAnalyzer::resolveJoin(std::shared_ptr&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x00000000283c086d 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::QueryAnalyzer::resolveQueryJoinTreeNode(std::shared_ptr&, DB::IdentifierResolveScope&, DB::QueryExpressionsAliasVisitor&) @ 0x00000000283ba42e 6. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ad09 7. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 8. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 9. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 12. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 13. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 15. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 16. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 17. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 21. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 22. asan_thread_start(void*) @ 0x000000000aa43059 23. ? @ 0x00007f12cd75eac3 24. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:46.485405 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:46.551769 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:46.923945 [ 2992 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:47.052129 [ 2982 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:47.117688 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:47.343439 [ 3001 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:47.584308 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:47.608955 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:47.932168 [ 3012 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:48.072772 [ 3018 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:48.081257 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:48.106421 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:48.141763 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:48.346433 [ 3027 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:48.486375 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:48.547048 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:48.565152 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:48.935200 [ 3037 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:49.075416 [ 3047 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:49.251447 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:49.348290 [ 3055 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:49.365321 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:49.393539 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:49.911412 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:49.938466 [ 3076 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:50.077995 [ 3075 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:50.279333 [ 13129 ] {10e74d0c-a47b-4c5a-ac0f-4afeca5e0642} OptimizeRegularExpression: Analyze RegularExpression failed, got error: Code: 306. DB::Exception: Stack size too large. Stack address: 0x7f0d3674c000, frame address: 0x7f0d36b4be90, stack size: 4194672, maximum stack size: 8388608. (TOO_DEEP_RECURSION) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:50.353927 [ 3088 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:50.488256 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:50.510505 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:50.659314 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:50.691418 [ 13129 ] {10e74d0c-a47b-4c5a-ac0f-4afeca5e0642} executeQuery: Code: 427. DB::Exception: OptimizedRegularExpression: cannot compile re2: ((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((, error: missing ): ((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((. Look at https://github.com/google/re2/wiki/Syntax for reference. Please note that if you specify regex as an SQL string literal, the slashes have to be additionally escaped. For example, to match an opening brace, write '\(' -- the first slash is for SQL and the second one is for regex: In scope SELECT match('', repeat('(', 100000)). (CANNOT_COMPILE_REGEXP) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37258) (comment: 02831_regexp_analyze_recursion.sql) (in query: SELECT match('', repeat('(', 100000));), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String const&) @ 0x000000000aab393a 4. ./build_docker/./src/Common/OptimizedRegularExpression.cpp:508: OptimizedRegularExpression::OptimizedRegularExpression(String const&, int) @ 0x000000001ae083d2 5. std::shared_ptr DB::Regexps::LocalCacheTable::getOrSet(String const&) @ 0x00000000143eaff9 6. DB::FunctionsStringSearch, (DB::ExecutionErrorPolicy)1, (DB::HaystackNeedleOrderIsConfigurable)0>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x00000000143e5261 7. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 8. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 10. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 11. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 12. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 13. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 20. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 21. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 24. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 25. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 27. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 28. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 29. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 2025.09.25 21:25:50.700933 [ 13129 ] {10e74d0c-a47b-4c5a-ac0f-4afeca5e0642} TCPHandler: Code: 427. DB::Exception: OptimizedRegularExpression: cannot compile re2: ((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((, error: missing ): ((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((. Look at https://github.com/google/re2/wiki/Syntax for reference. Please note that if you specify regex as an SQL string literal, the slashes have to be additionally escaped. For example, to match an opening brace, write '\(' -- the first slash is for SQL and the second one is for regex: In scope SELECT match('', repeat('(', 100000)). (CANNOT_COMPILE_REGEXP), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String const&) @ 0x000000000aab393a 4. ./build_docker/./src/Common/OptimizedRegularExpression.cpp:508: OptimizedRegularExpression::OptimizedRegularExpression(String const&, int) @ 0x000000001ae083d2 5. std::shared_ptr DB::Regexps::LocalCacheTable::getOrSet(String const&) @ 0x00000000143eaff9 6. DB::FunctionsStringSearch, (DB::ExecutionErrorPolicy)1, (DB::HaystackNeedleOrderIsConfigurable)0>::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x00000000143e5261 7. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 8. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 10. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 11. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 12. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 13. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 14. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 20. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 21. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 23. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 24. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 25. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 26. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 27. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 28. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 29. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 2025.09.25 21:25:50.945217 [ 3102 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:51.081830 [ 3106 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:51.243916 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:51.356896 [ 3113 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:51.429254 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:51.454479 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:51.810333 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:51.948180 [ 3119 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:52.069831 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:52.084778 [ 3134 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:52.459468 [ 3138 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:52.465875 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:52.487318 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:52.639922 [ 3359 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {df97ab97-a5cf-4ea8-bd10-026470f5fb5c::202509_1_166_33}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:52.697933 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:52.951435 [ 3142 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:53.001024 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:53.087891 [ 3157 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:53.308791 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:53.355982 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:53.377845 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:53.462611 [ 3183 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:53.954410 [ 3097 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:54.091161 [ 3191 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:54.380334 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:54.467247 [ 3179 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:54.658970 [ 13129 ] {d645d67f-162f-4ca8-9fa9-0572613eeeba} executeQuery: Code: 36. DB::Exception: Unexpected value of maximum unit argument (second) for function formatReadableTimeDelta, the only allowed values are: 'nanoseconds', 'microseconds', 'nanoseconds', 'seconds', 'minutes', 'hours', 'days', 'months', 'years'.: In scope SELECT formatReadableTimeDelta(-(((((1 + 60) + 3600) + 86400) + (30.5 * 86400)) + (365 * 86400)), 'second'). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37272) (comment: 01521_format_readable_time_delta2.sql) (in query: SELECT formatReadableTimeDelta(-(1 + 60 + 3600 + 86400 + 30.5 * 86400 + 365 * 86400), 'second');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception> const&, String>(int, FormatStringHelperImpl::type, std::type_identity> const&>::type, std::type_identity::type>, String const&, std::basic_string_view> const&, String&&) @ 0x000000000fdd9480 4. DB::(anonymous namespace)::FunctionFormatReadableTimeDelta::dispatchUnit(std::basic_string_view> const&, DB::(anonymous namespace)::FunctionFormatReadableTimeDelta::Unit, String const&) const @ 0x000000000fdd82a5 5. DB::(anonymous namespace)::FunctionFormatReadableTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000fdd5851 6. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 7. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 9. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 10. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 11. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 12. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 23. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 28. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 2025.09.25 21:25:54.662553 [ 13129 ] {d645d67f-162f-4ca8-9fa9-0572613eeeba} TCPHandler: Code: 36. DB::Exception: Unexpected value of maximum unit argument (second) for function formatReadableTimeDelta, the only allowed values are: 'nanoseconds', 'microseconds', 'nanoseconds', 'seconds', 'minutes', 'hours', 'days', 'months', 'years'.: In scope SELECT formatReadableTimeDelta(-(((((1 + 60) + 3600) + 86400) + (30.5 * 86400)) + (365 * 86400)), 'second'). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception> const&, String>(int, FormatStringHelperImpl::type, std::type_identity> const&>::type, std::type_identity::type>, String const&, std::basic_string_view> const&, String&&) @ 0x000000000fdd9480 4. DB::(anonymous namespace)::FunctionFormatReadableTimeDelta::dispatchUnit(std::basic_string_view> const&, DB::(anonymous namespace)::FunctionFormatReadableTimeDelta::Unit, String const&) const @ 0x000000000fdd82a5 5. DB::(anonymous namespace)::FunctionFormatReadableTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000fdd5851 6. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 7. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 8. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 9. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 10. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 11. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 12. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 19. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 20. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 22. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 23. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 24. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 27. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 28. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 29. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 31. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 2025.09.25 21:25:54.704381 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:54.707091 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:54.744084 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:54.927098 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:54.957511 [ 3190 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:55.094905 [ 3221 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:55.469960 [ 3236 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:55.488010 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:55.861090 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:55.949680 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:55.960337 [ 3246 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:55.965572 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:56.192292 [ 3230 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:56.472939 [ 3234 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:56.587419 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:56.968190 [ 2754 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:56.984156 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:57.139711 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:57.179666 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:57.195135 [ 2748 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:57.476506 [ 2774 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:57.549027 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:57.578571 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:57.636827 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:57.970550 [ 2758 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:58.043144 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:58.197856 [ 2787 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:58.262133 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:58.370554 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:58.482213 [ 2804 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:58.668305 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:58.973466 [ 2802 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:59.057917 [ 13110 ] {test_5_test_02585_query_to_kill_id_1} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 60.066363712 seconds, maximum: 60. (TIMEOUT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:54770) (comment: 02585_query_status_deadlock.sh) (in query: create temporary table tmp as select * from numbers(500000000);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002785801a 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000027856784 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d3463e7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:59.121617 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:59.133589 [ 13129 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:59.139041 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:59.200759 [ 2808 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:59.270048 [ 13110 ] {test_5_test_02585_query_to_kill_id_1} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 60.066363712 seconds, maximum: 60. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002785801a 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000027856784 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:57: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::CompletedPipelineExecutor::execute()::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d3463e7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:25:59.484894 [ 2811 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:59.782584 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:25:59.979250 [ 2833 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:00.118433 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:00.153794 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:00.228683 [ 2839 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:00.488247 [ 2841 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:00.874677 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:00.901303 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:00.930807 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:00.981461 [ 2864 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:01.232491 [ 2865 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:01.491389 [ 2868 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:01.741382 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:01.777192 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:01.984055 [ 2883 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:02.023370 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:02.235479 [ 2893 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:02.357795 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:02.430041 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:02.494728 [ 2906 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:02.784208 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:02.803414 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:02.987053 [ 2913 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:03.226513 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:03.238411 [ 2926 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:03.376564 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:03.424704 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:03.497666 [ 2944 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:03.627316 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:03.909476 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:03.939739 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:03.989535 [ 2943 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:04.158097 [ 13129 ] {9d5e28e8-cf65-4500-be76-90bc2a7b3fb1} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=733). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:59496) (comment: 02417_opentelemetry_insert_on_distributed_table.sh) (in query: select lower(hex(generateUUIDv4()))), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:26:04.190380 [ 13129 ] {9d5e28e8-cf65-4500-be76-90bc2a7b3fb1} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=733). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:26:04.241251 [ 2953 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:04.459633 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:04.500471 [ 2956 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:04.616201 [ 10192 ] {7fc0cc12-fe3b-4b97-b09c-ed4c71a97b58} executeQuery: Code: 62. DB::Exception: Syntax error: failed at position 118 ('-'): -insert-writeToLocal VALUES(1),(2). Expected one of: token sequence, Dot, token, OpeningRoundBracket. (SYNTAX_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:57648) (comment: 02417_opentelemetry_insert_on_distributed_table.sh) (in query: INSERT INTO test_c8uhfy7f.dist_opentelemetry SETTINGS distributed_foreground_insert=1, prefer_localhost_replica=async-insert-writeToLocal VALUES(1),(2)), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000000aaee710 3. ./build_docker/./src/Parsers/parseQuery.cpp:407: DB::parseQueryAndMovePosition(DB::IParser&, char const*&, char const*, String const&, bool, unsigned long, unsigned long, unsigned long) @ 0x000000002e9e8d35 4. ./build_docker/./src/Parsers/parseQuery.cpp:420: DB::parseQuery(DB::IParser&, char const*, char const*, String const&, unsigned long, unsigned long, unsigned long) @ 0x000000002e9e8e45 5. ./build_docker/./src/Interpreters/executeQuery.cpp:784: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3318a 6. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 7. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 8. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 9. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 13. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 14. asan_thread_start(void*) @ 0x000000000aa43059 15. ? @ 0x00007f12cd75eac3 16. ? @ 0x00007f12cd7f0850 2025.09.25 21:26:04.620742 [ 10192 ] {7fc0cc12-fe3b-4b97-b09c-ed4c71a97b58} DynamicQueryHandler: Code: 62. DB::Exception: Syntax error: failed at position 118 ('-'): -insert-writeToLocal VALUES(1),(2). Expected one of: token sequence, Dot, token, OpeningRoundBracket. (SYNTAX_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000000aaee710 3. ./build_docker/./src/Parsers/parseQuery.cpp:407: DB::parseQueryAndMovePosition(DB::IParser&, char const*&, char const*, String const&, bool, unsigned long, unsigned long, unsigned long) @ 0x000000002e9e8d35 4. ./build_docker/./src/Parsers/parseQuery.cpp:420: DB::parseQuery(DB::IParser&, char const*, char const*, String const&, unsigned long, unsigned long, unsigned long) @ 0x000000002e9e8e45 5. ./build_docker/./src/Interpreters/executeQuery.cpp:784: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3318a 6. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 7. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 8. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 9. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 10. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 12. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 13. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 14. asan_thread_start(void*) @ 0x000000000aa43059 15. ? @ 0x00007f12cd75eac3 16. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:04.779235 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:04.815045 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:04.991372 [ 2971 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:05.274023 [ 2968 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:05.403545 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:05.502563 [ 2984 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:05.738739 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:05.775109 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:05.994609 [ 2996 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:06.146965 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:06.197533 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:06.239991 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:06.275710 [ 3020 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:06.511252 [ 3014 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:06.646175 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:06.679682 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:06.998081 [ 3034 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:07.164078 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:07.278869 [ 3039 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:07.514580 [ 3052 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:07.960150 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:08.001608 [ 3061 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:08.043134 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:08.282599 [ 3074 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:08.330759 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:08.489159 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:08.515460 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:08.517418 [ 3088 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:08.973998 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:09.004961 [ 3094 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:09.286847 [ 3113 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:09.322760 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:09.520420 [ 3117 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:09.620230 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:09.645483 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:10.009216 [ 3127 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:10.289943 [ 3148 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:10.300204 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:10.396425 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:10.412766 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:10.478233 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:10.523293 [ 3140 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:10.590182 [ 13129 ] {4ebb6767-1b15-401b-8ac2-13c03619085b} executeQuery: Code: 32. DB::Exception: Attempt to read after eof: while converting '' to Enum8('INTERNAL' = 0, 'SERVER' = 1, 'CLIENT' = 2, 'PRODUCER' = 3, 'CONSUMER' = 4). (ATTEMPT_TO_READ_AFTER_EOF) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48556) (comment: 02417_opentelemetry_insert_on_distributed_table.sh) (in query: SELECT count() FROM system.opentelemetry_span_log WHERE finish_date >= yesterday() AND lower(hex(trace_id)) = 'SERVER' AND kind = '' ;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/IO/VarInt.cpp:13: DB::throwReadAfterEOF() @ 0x000000001aefe9cb 5. void DB::readIntTextImpl<_BitInt(8), void, (DB::ReadIntTextCheckOverflow)0>(_BitInt(8)&, DB::ReadBuffer&) @ 0x000000000b5555d0 6. ./src/IO/BufferBase.h:41: DB::EnumValues<_BitInt(8)>::getValue(StringRef, bool) const @ 0x00000000279e6be9 7. ./build_docker/./src/DataTypes/Serializations/SerializationEnum.cpp:0: DB::SerializationEnum<_BitInt(8)>::deserializeWholeText(DB::IColumn&, DB::ReadBuffer&, DB::FormatSettings const&) const @ 0x0000000027b1c318 8. ./build_docker/./src/Interpreters/convertFieldToType.cpp:554: DB::(anonymous namespace)::convertFieldToTypeImpl(DB::Field const&, DB::IDataType const&, DB::IDataType const*) @ 0x0000000029aa606f 9. ./build_docker/./src/Interpreters/convertFieldToType.cpp:611: DB::convertFieldToType(DB::Field const&, DB::IDataType const&, DB::IDataType const*) @ 0x0000000029aa2141 10. DB::FunctionComparison::executeWithConstString(std::shared_ptr const&, DB::IColumn const*, DB::IColumn const*, std::shared_ptr const&, std::shared_ptr const&, unsigned long) const @ 0x000000000f345710 11. DB::FunctionComparison::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000f3133d1 12. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 13. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 14. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 15. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 16. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 17. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::ActionsDAG::evaluatePartialResult(std::unordered_map, std::equal_to, std::allocator>>&, std::vector> const&, unsigned long, bool) @ 0x0000000028259428 18. ./contrib/llvm-project/libcxx/include/vector:961: DB::ActionsDAG::updateHeader(DB::Block const&) const @ 0x0000000028256e17 19. ./build_docker/./src/Processors/Transforms/FilterTransform.cpp:0: DB::FilterTransform::transformHeader(DB::Block const&, DB::ActionsDAG const*, String const&, bool) @ 0x000000002d9c9921 20. ./build_docker/./src/Processors/QueryPlan/FilterStep.cpp:0: DB::FilterStep::FilterStep(DB::DataStream const&, DB::ActionsDAG, String, bool) @ 0x000000002ddb09e1 21. ./contrib/llvm-project/libcxx/include/string:1499: std::__unique_if::__unique_single std::make_unique[abi:v15007](DB::DataStream const&, DB::ActionsDAG&&, String&, bool&) @ 0x00000000293c12e7 22. ./build_docker/./src/Planner/Planner.cpp:0: DB::(anonymous namespace)::addFilterStep(DB::QueryPlan&, DB::FilterAnalysisResult&, String const&, std::unordered_set, std::hash>, std::equal_to>, std::allocator>>&) @ 0x000000002943b3f5 23. ./contrib/llvm-project/libcxx/include/string:1499: DB::Planner::buildPlanForQueryNode() @ 0x000000002942e6e9 24. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 25. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 27. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 28. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 29. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 2025.09.25 21:26:10.609767 [ 13129 ] {4ebb6767-1b15-401b-8ac2-13c03619085b} TCPHandler: Code: 32. DB::Exception: Attempt to read after eof: while converting '' to Enum8('INTERNAL' = 0, 'SERVER' = 1, 'CLIENT' = 2, 'PRODUCER' = 3, 'CONSUMER' = 4). (ATTEMPT_TO_READ_AFTER_EOF), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/IO/VarInt.cpp:13: DB::throwReadAfterEOF() @ 0x000000001aefe9cb 5. void DB::readIntTextImpl<_BitInt(8), void, (DB::ReadIntTextCheckOverflow)0>(_BitInt(8)&, DB::ReadBuffer&) @ 0x000000000b5555d0 6. ./src/IO/BufferBase.h:41: DB::EnumValues<_BitInt(8)>::getValue(StringRef, bool) const @ 0x00000000279e6be9 7. ./build_docker/./src/DataTypes/Serializations/SerializationEnum.cpp:0: DB::SerializationEnum<_BitInt(8)>::deserializeWholeText(DB::IColumn&, DB::ReadBuffer&, DB::FormatSettings const&) const @ 0x0000000027b1c318 8. ./build_docker/./src/Interpreters/convertFieldToType.cpp:554: DB::(anonymous namespace)::convertFieldToTypeImpl(DB::Field const&, DB::IDataType const&, DB::IDataType const*) @ 0x0000000029aa606f 9. ./build_docker/./src/Interpreters/convertFieldToType.cpp:611: DB::convertFieldToType(DB::Field const&, DB::IDataType const&, DB::IDataType const*) @ 0x0000000029aa2141 10. DB::FunctionComparison::executeWithConstString(std::shared_ptr const&, DB::IColumn const*, DB::IColumn const*, std::shared_ptr const&, std::shared_ptr const&, unsigned long) const @ 0x000000000f345710 11. DB::FunctionComparison::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000f3133d1 12. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 13. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 14. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 15. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 16. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 17. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::ActionsDAG::evaluatePartialResult(std::unordered_map, std::equal_to, std::allocator>>&, std::vector> const&, unsigned long, bool) @ 0x0000000028259428 18. ./contrib/llvm-project/libcxx/include/vector:961: DB::ActionsDAG::updateHeader(DB::Block const&) const @ 0x0000000028256e17 19. ./build_docker/./src/Processors/Transforms/FilterTransform.cpp:0: DB::FilterTransform::transformHeader(DB::Block const&, DB::ActionsDAG const*, String const&, bool) @ 0x000000002d9c9921 20. ./build_docker/./src/Processors/QueryPlan/FilterStep.cpp:0: DB::FilterStep::FilterStep(DB::DataStream const&, DB::ActionsDAG, String, bool) @ 0x000000002ddb09e1 21. ./contrib/llvm-project/libcxx/include/string:1499: std::__unique_if::__unique_single std::make_unique[abi:v15007](DB::DataStream const&, DB::ActionsDAG&&, String&, bool&) @ 0x00000000293c12e7 22. ./build_docker/./src/Planner/Planner.cpp:0: DB::(anonymous namespace)::addFilterStep(DB::QueryPlan&, DB::FilterAnalysisResult&, String const&, std::unordered_set, std::hash>, std::equal_to>, std::allocator>>&) @ 0x000000002943b3f5 23. ./contrib/llvm-project/libcxx/include/string:1499: DB::Planner::buildPlanForQueryNode() @ 0x000000002942e6e9 24. ./build_docker/./src/Planner/Planner.cpp:1254: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000294261fb 25. ./src/Planner/Planner.h:44: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x0000000029421552 26. ./build_docker/./src/Interpreters/executeQuery.cpp:1182: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3737e 27. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 28. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 29. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 30. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 31. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 2025.09.25 21:26:10.813678 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:10.843898 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:11.017130 [ 3153 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:11.141543 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:11.292597 [ 3173 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:11.560825 [ 3183 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:11.792777 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:11.810226 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:11.822076 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:12.019625 [ 3186 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:12.295530 [ 3199 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:12.381813 [ 13129 ] {096e355c-d0f9-4326-bdab-beaa3d6cb432} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48598) (comment: 02417_opentelemetry_insert_on_distributed_table.sh) (in query: SYSTEM FLUSH DISTRIBUTED test_c8uhfy7f.dist_opentelemetry), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002b74efa8 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageDistributed::flushClusterNodesAllData(std::shared_ptr, DB::SettingsChanges const&) @ 0x000000002b7506be 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSystemQuery::flushDistributed(DB::ASTSystemQuery&) @ 0x0000000029bb0fe3 10. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:697: DB::InterpreterSystemQuery::execute() @ 0x0000000029b98552 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 14. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 18. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 2025.09.25 21:26:12.403641 [ 13129 ] {096e355c-d0f9-4326-bdab-beaa3d6cb432} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=0, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002b74efa8 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageDistributed::flushClusterNodesAllData(std::shared_ptr, DB::SettingsChanges const&) @ 0x000000002b7506be 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterSystemQuery::flushDistributed(DB::ASTSystemQuery&) @ 0x0000000029bb0fe3 10. ./build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:697: DB::InterpreterSystemQuery::execute() @ 0x0000000029b98552 11. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 14. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 16. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 18. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 19. asan_thread_start(void*) @ 0x000000000aa43059 20. ? @ 0x00007f12cd75eac3 21. ? @ 0x00007f12cd7f0850 2025.09.25 21:26:12.563169 [ 3179 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:12.693925 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:12.784670 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:12.821650 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:13.021780 [ 3213 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:13.156567 [ 3359 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {f09856a1-8272-4b8d-a57a-2bc74603fb75::202509_1_30_6}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:13.297705 [ 3241 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:13.565710 [ 3227 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:13.665879 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:13.776317 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:13.789922 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:14.024652 [ 3230 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:14.224006 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:14.237333 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:14.300164 [ 3254 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:14.441346 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:14.568097 [ 2761 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:15.027214 [ 2766 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:15.190016 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:15.195290 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:15.224956 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:15.302336 [ 2789 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:15.570542 [ 2792 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:16.054480 [ 2790 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:16.207050 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:16.217722 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:16.225410 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:16.304616 [ 2827 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:16.573613 [ 2828 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:16.926178 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:16.992770 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:17.007233 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:17.056659 [ 2843 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:17.139295 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:17.306869 [ 2846 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:17.538700 [ 13130 ] {dbe110a8-bf37-42d0-8e5e-87b445e2f659} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48632) (comment: 02417_opentelemetry_insert_on_distributed_table.sh) (in query: SELECT count() FROM system.opentelemetry_span_log WHERE finish_date >= yesterday() AND lower(hex(trace_id)) = 'bc3729b61ca4442697ee3d79a942483c' AND kind = 'SERVER' ;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(bool) const @ 0x000000002de3515e 10. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizeUseAggregateProjection.cpp:604: DB::QueryPlanOptimizations::optimizeUseAggregateProjections(DB::QueryPlan::Node&, std::list>&, bool) @ 0x000000002dfa7c7f 11. ./contrib/llvm-project/libcxx/include/optional:344: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list>&) @ 0x000000002df631d6 12. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:507: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x000000002de001ca 13. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:36: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddfec4b 14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 17. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 19. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 23. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 24. asan_thread_start(void*) @ 0x000000000aa43059 25. ? @ 0x00007f12cd75eac3 26. ? @ 0x00007f12cd7f0850 2025.09.25 21:26:17.543872 [ 13130 ] {dbe110a8-bf37-42d0-8e5e-87b445e2f659} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=1, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af82f33 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c64fc80 7. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1723: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, std::shared_ptr, std::shared_ptr const&, DB::SelectQueryInfo const&, std::shared_ptr, unsigned long, std::shared_ptr, std::equal_to, std::allocator>>>, DB::MergeTreeData const&, std::vector> const&, std::shared_ptr, std::optional&, bool) @ 0x000000002de379fd 8. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1437: DB::ReadFromMergeTree::selectRangesToRead(std::vector, std::allocator>>, bool) const @ 0x000000002de3554f 9. ./build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:0: DB::ReadFromMergeTree::selectRangesToRead(bool) const @ 0x000000002de3515e 10. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizeUseAggregateProjection.cpp:604: DB::QueryPlanOptimizations::optimizeUseAggregateProjections(DB::QueryPlan::Node&, std::list>&, bool) @ 0x000000002dfa7c7f 11. ./contrib/llvm-project/libcxx/include/optional:344: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list>&) @ 0x000000002df631d6 12. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:507: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x000000002de001ca 13. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:36: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddfec4b 14. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 15. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 16. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 17. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 19. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 20. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 21. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 22. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 23. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 24. asan_thread_start(void*) @ 0x000000000aa43059 25. ? @ 0x00007f12cd75eac3 26. ? @ 0x00007f12cd7f0850 2025.09.25 21:26:17.579020 [ 2862 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:18.002280 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:18.048985 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:18.060920 [ 2867 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:18.184137 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:18.314426 [ 2887 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:18.581703 [ 2837 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:18.765922 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:18.802593 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:19.063651 [ 2899 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:19.238525 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:19.316355 [ 2919 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:19.496249 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:19.584216 [ 2927 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:19.584532 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:19.926090 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:20.069207 [ 2942 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:20.267488 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:20.318870 [ 2937 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:20.586850 [ 2952 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:20.791235 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:20.836824 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:20.992899 [ 10192 ] {653589d9-8220-4a17-b247-a8681023896b} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=1). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:37376) (comment: 02417_opentelemetry_insert_on_distributed_table.sh) (in query: INSERT INTO test_c8uhfy7f.dist_opentelemetry SETTINGS distributed_foreground_insert=1, prefer_localhost_replica=1 VALUES), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c210b10 7. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function, DB::ThreadStatus*, std::atomic*) @ 0x000000002d9c1551 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d9c0837 9. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 10. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 13. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:113: DB::CompletedPipelineExecutor::execute() @ 0x000000002d34579e 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1635: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b40110 15. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 16. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 17. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 21. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 22. asan_thread_start(void*) @ 0x000000000aa43059 23. ? @ 0x00007f12cd75eac3 24. ? @ 0x00007f12cd7f0850 2025.09.25 21:26:21.071314 [ 2966 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:21.096704 [ 10192 ] {653589d9-8220-4a17-b247-a8681023896b} DynamicQueryHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=2, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c210b10 7. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function, DB::ThreadStatus*, std::atomic*) @ 0x000000002d9c1551 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d9c0837 9. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 10. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 11. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 13. ./build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:113: DB::CompletedPipelineExecutor::execute() @ 0x000000002d34579e 14. ./build_docker/./src/Interpreters/executeQuery.cpp:1635: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b40110 15. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 16. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 17. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 18. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 19. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 20. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 21. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 22. asan_thread_start(void*) @ 0x000000000aa43059 23. ? @ 0x00007f12cd75eac3 24. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:21.188713 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:21.321455 [ 2971 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:21.588771 [ 2984 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:21.611929 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:21.657916 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:21.961335 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:22.040961 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:22.063473 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:22.073511 [ 2933 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:22.324016 [ 3011 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:22.426048 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:22.477566 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:22.591364 [ 3016 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:22.763570 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:22.935140 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:23.076061 [ 3039 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:23.328625 [ 3043 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:23.593452 [ 3058 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:23.598149 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:23.642673 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:23.943390 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:24.079078 [ 3078 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:24.305045 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:24.330899 [ 3091 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:24.595611 [ 3099 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:24.893435 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:24.934945 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:25.081052 [ 3117 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:25.294041 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:25.334881 [ 3124 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:25.530724 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:25.597889 [ 3151 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:25.787863 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:25.820672 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:26.084398 [ 3155 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:26.123937 [ 13130 ] {4e6d5552-6281-4d37-860b-d8be6cf98281} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=0): (while reading column query_id): (while reading from part /var/lib/clickhouse/store/f81/f81dbc1c-a1bd-4968-9514-f0c97137169a/202509_1_143_30/ in table system.query_log (f81dbc1c-a1bd-4968-9514-f0c97137169a) located on disk default of type local, from mark 0 with max_rows_to_read = 8071): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60888) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x000000000b581216 13. ./build_docker/./src/Compression/CompressedReadBufferBase.cpp:0: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x000000002775f390 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::CachedCompressedReadBuffer::nextImpl() @ 0x000000002c7f8e5d 15. DB::ReadBuffer::next() @ 0x000000000b528aeb 16. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:159: void DB::deserializeBinarySSE2<1>(DB::PODArray, 63ul, 64ul>&, DB::PODArray, 63ul, 64ul>&, DB::ReadBuffer&, unsigned long) @ 0x0000000027c11390 17. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:259: DB::SerializationString::deserializeBinaryBulk(DB::IColumn&, DB::ReadBuffer&, unsigned long, double) const @ 0x0000000027c0ec00 18. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027aa00db 19. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:196: DB::MergeTreeReaderWide::readData(DB::NameAndTypePair const&, std::shared_ptr const&, COW::immutable_ptr&, unsigned long, bool, unsigned long, unsigned long, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>&, bool) @ 0x000000002c80211e 20. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:158: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c800e8d 21. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 22. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000002c7e0473 23. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbec3 24. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbd35 25. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 26. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:53: DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002dea5374 27. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 28. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 29. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 30. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 31. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 2025.09.25 21:26:26.227672 [ 13130 ] {4e6d5552-6281-4d37-860b-d8be6cf98281} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=0): (while reading column query_id): (while reading from part /var/lib/clickhouse/store/f81/f81dbc1c-a1bd-4968-9514-f0c97137169a/202509_1_143_30/ in table system.query_log (f81dbc1c-a1bd-4968-9514-f0c97137169a) located on disk default of type local, from mark 0 with max_rows_to_read = 8071): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x000000000b581216 13. ./build_docker/./src/Compression/CompressedReadBufferBase.cpp:0: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x000000002775f390 14. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::CachedCompressedReadBuffer::nextImpl() @ 0x000000002c7f8e5d 15. DB::ReadBuffer::next() @ 0x000000000b528aeb 16. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:159: void DB::deserializeBinarySSE2<1>(DB::PODArray, 63ul, 64ul>&, DB::PODArray, 63ul, 64ul>&, DB::ReadBuffer&, unsigned long) @ 0x0000000027c11390 17. ./build_docker/./src/DataTypes/Serializations/SerializationString.cpp:259: DB::SerializationString::deserializeBinaryBulk(DB::IColumn&, DB::ReadBuffer&, unsigned long, double) const @ 0x0000000027c0ec00 18. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027aa00db 19. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:196: DB::MergeTreeReaderWide::readData(DB::NameAndTypePair const&, std::shared_ptr const&, COW::immutable_ptr&, unsigned long, bool, unsigned long, unsigned long, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>&, bool) @ 0x000000002c80211e 20. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:158: DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c800e8d 21. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 22. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000002c7e0473 23. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbec3 24. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbd35 25. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 26. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:53: DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002dea5374 27. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 28. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 29. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 30. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 31. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 2025.09.25 21:26:26.339607 [ 3164 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:26.376011 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:26.498467 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:26.601215 [ 3165 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:26.641957 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:26.895271 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:26.900349 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:26.921140 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:27.087472 [ 3191 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:27.342448 [ 3202 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:27.607573 [ 3192 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:27.950455 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:28.091102 [ 3208 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:28.197351 [ 13129 ] {fd648b2c-0e62-4412-be30-0b991a5044e8} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37284) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:26:28.205005 [ 13129 ] {fd648b2c-0e62-4412-be30-0b991a5044e8} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:26:28.345477 [ 3222 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:28.390628 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:28.430446 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:28.610983 [ 3227 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:28.681638 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:29.094319 [ 3246 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:29.239966 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:29.330320 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:29.347639 [ 3257 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:29.362912 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:29.613591 [ 2767 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:29.622523 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:29.667915 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:29.886845 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:30.097389 [ 2774 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:30.202939 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:30.253417 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:30.356328 [ 2777 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:30.607066 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:30.616550 [ 2787 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:30.749618 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:30.782009 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:31.100260 [ 2810 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:31.359474 [ 2804 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:31.493129 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:31.516176 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:31.622332 [ 2809 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:31.737523 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:32.103077 [ 2832 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:32.274828 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:32.364906 [ 2854 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:32.498812 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:32.531053 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:32.625642 [ 2855 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:33.106475 [ 2874 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:33.301791 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:33.367858 [ 2884 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:33.436490 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:33.457273 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:33.605744 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:33.628890 [ 2901 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:34.109628 [ 2916 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:34.370962 [ 2913 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:34.576202 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:34.632160 [ 2926 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:34.762642 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:34.786791 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:34.877232 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:35.119133 [ 2945 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:35.158697 [ 13129 ] {cc34f906-7b85-4c2b-baea-925e00085558} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37324) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:26:35.167494 [ 13129 ] {cc34f906-7b85-4c2b-baea-925e00085558} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:26:35.373006 [ 2953 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:35.635405 [ 2948 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:35.823429 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:35.825668 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:35.850696 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:36.059939 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:36.122807 [ 2968 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:36.377378 [ 2989 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:36.638511 [ 3000 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:36.648021 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:36.677007 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:36.686411 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:36.896834 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:37.041918 [ 13129 ] {e4b2ba80-1173-4f7f-aa28-1aa6ce9776eb} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58456) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:26:37.048875 [ 13129 ] {e4b2ba80-1173-4f7f-aa28-1aa6ce9776eb} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:26:37.125671 [ 3013 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:37.462801 [ 3026 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:37.486186 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:37.530241 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:37.641218 [ 3035 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:37.673671 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:37.917636 [ 3358 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {95e6584c-1281-4071-8ae7-ba1b29088860::202509_1_166_33}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:37.964184 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:38.130360 [ 3037 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:38.466058 [ 3048 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:38.644624 [ 3045 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:38.808132 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:38.904900 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:38.972728 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:39.133747 [ 3057 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:39.283110 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:39.367804 [ 13130 ] {d8399968-a4b2-4ab5-beb2-41772edf66ab} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58466) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:26:39.373660 [ 13130 ] {d8399968-a4b2-4ab5-beb2-41772edf66ab} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:26:39.469473 [ 3074 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:39.647798 [ 3091 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:39.684940 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:39.702179 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:39.779250 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:39.885220 [ 3352 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {95e6584c-1281-4071-8ae7-ba1b29088860::202509_1_166_33}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:40.137634 [ 3082 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:40.473519 [ 3101 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:40.474896 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:40.546277 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:40.651090 [ 3077 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:40.668485 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:41.046541 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:41.140814 [ 3121 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:41.476613 [ 3135 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:41.518223 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:41.541209 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:41.616513 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:41.654407 [ 3148 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:42.143826 [ 3149 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:42.301382 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:42.417327 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:42.419785 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:42.479915 [ 3160 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:42.657386 [ 3178 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:43.146508 [ 3180 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:43.185139 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:43.232213 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:43.395906 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:43.574261 [ 3207 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:43.660469 [ 3205 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:43.752571 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:43.783982 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:44.149851 [ 3219 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:44.190014 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:44.577537 [ 3236 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:44.666337 [ 3229 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:44.819998 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:44.821335 [ 10192 ] {91277681-ee62-4b0d-bba2-fdb777fc743c} executeQuery: Code: 36. DB::Exception: Explicitly specified value of minimum unit argument (hours) for function formatReadableTimeDelta must not be greater than maximum unit value (seconds).: In scope SELECT formatReadableTimeDelta(1.1, 'seconds', 'hours'). (BAD_ARGUMENTS) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58498) (comment: 02887_format_readable_timedelta_subseconds.sql) (in query: -- Check exception is thrown SELECT formatReadableTimeDelta(1.1, 'seconds', 'hours');), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&, String, std::basic_string_view>&>(int, FormatStringHelperImpl>&>::type, std::type_identity::type, std::type_identity>&>::type>, std::basic_string_view>&, String&&, std::basic_string_view>&) @ 0x000000000fdd9220 4. DB::(anonymous namespace)::FunctionFormatReadableTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000fdd67c7 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:26:44.830261 [ 10192 ] {91277681-ee62-4b0d-bba2-fdb777fc743c} TCPHandler: Code: 36. DB::Exception: Explicitly specified value of minimum unit argument (hours) for function formatReadableTimeDelta must not be greater than maximum unit value (seconds).: In scope SELECT formatReadableTimeDelta(1.1, 'seconds', 'hours'). (BAD_ARGUMENTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&, String, std::basic_string_view>&>(int, FormatStringHelperImpl>&>::type, std::type_identity::type, std::type_identity>&>::type>, std::basic_string_view>&, String&&, std::basic_string_view>&) @ 0x000000000fdd9220 4. DB::(anonymous namespace)::FunctionFormatReadableTimeDelta::executeImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000fdd67c7 5. DB::IFunction::executeImplDryRun(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aa8c635 6. DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::vector> const&, std::shared_ptr const&, unsigned long) const @ 0x000000000aac91d5 7. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccccfb 8. DB::IExecutableFunction::defaultImplementationForConstantArguments(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000cccc1ab 9. DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccccc2a 10. DB::IExecutableFunction::executeWithoutSparseColumns(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccce65c 11. DB::IExecutableFunction::execute(std::vector> const&, std::shared_ptr const&, unsigned long, bool) const @ 0x000000000ccd2030 12. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:117: DB::QueryAnalyzer::resolveFunction(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x000000002839a3af 13. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveExpressionNode(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool, bool) @ 0x0000000028367bce 14. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:3909: DB::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&, bool, bool) @ 0x0000000028366290 15. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:4160: DB::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr&, DB::IdentifierResolveScope&) @ 0x00000000283b62c9 16. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolveQuery(std::shared_ptr const&, DB::IdentifierResolveScope&) @ 0x000000002835ae03 17. ./build_docker/./src/Analyzer/Resolve/QueryAnalyzer.cpp:0: DB::QueryAnalyzer::resolve(std::shared_ptr&, std::shared_ptr const&, std::shared_ptr) @ 0x0000000028359104 18. ./build_docker/./src/Analyzer/Resolve/QueryAnalysisPass.cpp:19: DB::QueryAnalysisPass::run(std::shared_ptr&, std::shared_ptr) @ 0x0000000028357f0d 19. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:0: DB::QueryTreePassManager::run(std::shared_ptr) @ 0x000000002921bdcc 20. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr const&, DB::SelectQueryOptions const&, std::shared_ptr const&, std::shared_ptr const&) @ 0x000000002941f313 21. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:160: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr const&, std::shared_ptr const&, DB::SelectQueryOptions const&, std::vector> const&) @ 0x0000000029419aef 22. ./contrib/llvm-project/libcxx/include/vector:432: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, std::shared_ptr const&, DB::SelectQueryOptions const&>(std::shared_ptr&, std::shared_ptr const&, DB::SelectQueryOptions const&) @ 0x0000000029421f5d 23. ./build_docker/./src/Interpreters/InterpreterFactory.cpp:0: DB::InterpreterFactory::get(std::shared_ptr&, std::shared_ptr, DB::SelectQueryOptions const&) @ 0x0000000029344c1b 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3703b 25. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 27. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 28. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 30. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 31. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 2025.09.25 21:26:44.950051 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:45.055857 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:45.153696 [ 3249 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:45.579760 [ 3255 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:45.647088 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:45.668459 [ 3258 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:45.674166 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:45.963335 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:46.156446 [ 2761 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:46.329717 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:46.345886 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:46.466100 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:46.584675 [ 2780 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:46.671415 [ 2769 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:46.852868 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:46.871869 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:47.039676 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:47.159077 [ 2788 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:47.297281 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:47.656656 [ 2787 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:47.675194 [ 2784 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:47.817550 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:47.834169 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:48.072767 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:48.167612 [ 2817 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:48.569547 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:48.606287 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:48.674043 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:48.728582 [ 2809 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:48.729318 [ 2821 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:49.175839 [ 2822 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:49.225613 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:49.731457 [ 2824 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:49.732199 [ 2841 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:49.879613 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:49.895553 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:50.179505 [ 2850 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:50.296596 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:50.348760 [ 3350 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {68caad89-531d-40c0-87c3-4b3574cf49ff::202509_1_160_32}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:50.499168 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:50.539855 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:50.727405 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:50.733976 [ 2874 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:50.735498 [ 2876 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:51.047084 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:51.095348 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:51.181594 [ 2880 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:51.490783 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:51.513276 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:51.736803 [ 2908 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:51.737952 [ 2905 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:51.786991 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:52.183525 [ 2913 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:52.221380 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:52.229109 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:52.690303 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:52.715880 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:52.740067 [ 2944 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:52.740915 [ 2929 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:52.825907 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:53.186260 [ 2946 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:53.591781 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:53.620442 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:53.743263 [ 2959 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:53.743588 [ 2956 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:53.756110 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:54.066385 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:54.104080 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:54.195317 [ 2960 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:54.720868 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:54.746654 [ 2972 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:54.747686 [ 2986 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:54.749428 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:54.778874 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:55.197900 [ 2980 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:55.546788 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:55.749374 [ 2993 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:55.750862 [ 3005 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:55.947683 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:55.968976 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:56.124780 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:56.200669 [ 3002 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:56.499746 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:56.747019 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:56.753893 [ 3033 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:56.755281 [ 3023 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:56.778193 [ 3816 ] {2e024d60-d797-43d8-969c-ab8a5422ba1b} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::ffff:127.0.0.1]:37388) (comment: 02457_s3_cluster_schema_inference.sql) (in query: SELECT __table1.c1 AS c1, __table1.c2 AS c2, __table1.c3 AS c3 FROM s3Cluster('test_cluster_one_shard_three_replicas_localhost', 'http://localhost:11111/test/{a,b}.tsv', 'TSV', '`c1` Nullable(Int64), `c2` Nullable(Int64), `c3` Nullable(Int64)') AS __table1 ORDER BY __table1.c1 ASC, __table1.c2 ASC, __table1.c3 ASC), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000267fd682 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000267fd094 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000267e6c01 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000267d95d5 10. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:233: DB::(anonymous namespace)::ReadFromObjectStorageStep::createIterator(DB::ActionsDAG::Node const*) @ 0x00000000267c14a8 11. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:172: DB::(anonymous namespace)::ReadFromObjectStorageStep::applyFilters(DB::ActionDAGNodes) @ 0x00000000267c0ef8 12. ./contrib/llvm-project/libcxx/include/vector:590: DB::SourceStepWithFilter::applyFilters() @ 0x000000002df93a35 13. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizePrimaryKeyConditionAndLimit.cpp:0: DB::QueryPlanOptimizations::optimizePrimaryKeyConditionAndLimit(std::vector> const&) @ 0x000000002df9366a 14. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:122: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list>&) @ 0x000000002df62d06 15. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:507: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x000000002de001ca 16. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:36: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddfec4b 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 19. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 20. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 22. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 26. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 27. asan_thread_start(void*) @ 0x000000000aa43059 28. ? @ 0x00007f12cd75eac3 29. ? @ 0x00007f12cd7f0850 2025.09.25 21:26:56.787415 [ 3816 ] {2e024d60-d797-43d8-969c-ab8a5422ba1b} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: failed to start the thread (threads=2, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000267fd682 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000267fd094 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000267e6c01 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000267d95d5 10. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:233: DB::(anonymous namespace)::ReadFromObjectStorageStep::createIterator(DB::ActionsDAG::Node const*) @ 0x00000000267c14a8 11. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:172: DB::(anonymous namespace)::ReadFromObjectStorageStep::applyFilters(DB::ActionDAGNodes) @ 0x00000000267c0ef8 12. ./contrib/llvm-project/libcxx/include/vector:590: DB::SourceStepWithFilter::applyFilters() @ 0x000000002df93a35 13. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizePrimaryKeyConditionAndLimit.cpp:0: DB::QueryPlanOptimizations::optimizePrimaryKeyConditionAndLimit(std::vector> const&) @ 0x000000002df9366a 14. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:122: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list>&) @ 0x000000002df62d06 15. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:507: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x000000002de001ca 16. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:36: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddfec4b 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 19. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 20. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 22. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 26. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 27. asan_thread_start(void*) @ 0x000000000aa43059 28. ? @ 0x00007f12cd75eac3 29. ? @ 0x00007f12cd7f0850 2025.09.25 21:26:56.827363 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:56.860923 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:56.888927 [ 10192 ] {ce23b99b-477a-4e89-9373-bf374c60209b} executeQuery: Code: 439. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: Cannot schedule a task: failed to start the thread (threads=2, jobs=2). Stack trace: 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000267fd682 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000267fd094 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000267e6c01 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000267d95d5 10. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:233: DB::(anonymous namespace)::ReadFromObjectStorageStep::createIterator(DB::ActionsDAG::Node const*) @ 0x00000000267c14a8 11. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:172: DB::(anonymous namespace)::ReadFromObjectStorageStep::applyFilters(DB::ActionDAGNodes) @ 0x00000000267c0ef8 12. ./contrib/llvm-project/libcxx/include/vector:590: DB::SourceStepWithFilter::applyFilters() @ 0x000000002df93a35 13. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizePrimaryKeyConditionAndLimit.cpp:0: DB::QueryPlanOptimizations::optimizePrimaryKeyConditionAndLimit(std::vector> const&) @ 0x000000002df9366a 14. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:122: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list>&) @ 0x000000002df62d06 15. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:507: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x000000002de001ca 16. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:36: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddfec4b 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 19. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 20. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 22. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 26. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 27. asan_thread_start(void*) @ 0x000000000aa43059 28. ? @ 0x00007f12cd75eac3 29. ? @ 0x00007f12cd7f0850 : While executing Remote. (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:46532) (comment: 02457_s3_cluster_schema_inference.sql) (in query: select * from s3Cluster('test_cluster_one_shard_three_replicas_localhost', 'http://localhost:11111/test/{a,b}.tsv') order by c1, c2, c3;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001aeb974e 3. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000002cee6c6a 4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x000000002cef780e 5. ./build_docker/./src/Client/MultiplexedConnections.cpp:340: DB::MultiplexedConnections::receivePacketUnlocked(std::function) @ 0x000000002cf8e09e 6. ./build_docker/./src/QueryPipeline/RemoteQueryExecutorReadContext.cpp:0: DB::RemoteQueryExecutorReadContext::Task::run(std::function, std::function) @ 0x00000000278ad616 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278ab436 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278aaa44 9. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func>>(unsigned int, unsigned int) @ 0x00000000278a8394 2025.09.25 21:26:56.895275 [ 10192 ] {ce23b99b-477a-4e89-9373-bf374c60209b} TCPHandler: Code: 439. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: Cannot schedule a task: failed to start the thread (threads=2, jobs=2). Stack trace: 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./build_docker/./src/Common/ThreadPool.cpp:0: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af819ed 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000267fd682 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000267fd094 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000267e6c01 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000267d95d5 10. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:233: DB::(anonymous namespace)::ReadFromObjectStorageStep::createIterator(DB::ActionsDAG::Node const*) @ 0x00000000267c14a8 11. ./build_docker/./src/Storages/ObjectStorage/StorageObjectStorage.cpp:172: DB::(anonymous namespace)::ReadFromObjectStorageStep::applyFilters(DB::ActionDAGNodes) @ 0x00000000267c0ef8 12. ./contrib/llvm-project/libcxx/include/vector:590: DB::SourceStepWithFilter::applyFilters() @ 0x000000002df93a35 13. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizePrimaryKeyConditionAndLimit.cpp:0: DB::QueryPlanOptimizations::optimizePrimaryKeyConditionAndLimit(std::vector> const&) @ 0x000000002df9366a 14. ./build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:122: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list>&) @ 0x000000002df62d06 15. ./build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:507: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x000000002de001ca 16. ./contrib/llvm-project/libcxx/include/__memory/compressed_pair.h:36: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) @ 0x000000002ddfec4b 17. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:252: DB::InterpreterSelectQueryAnalyzer::buildQueryPipeline() @ 0x000000002942117f 18. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:436: DB::InterpreterSelectQueryAnalyzer::execute() @ 0x00000000294209e7 19. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 20. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b31685 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d17616c 22. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 23. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 24. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 25. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 26. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 27. asan_thread_start(void*) @ 0x000000000aa43059 28. ? @ 0x00007f12cd75eac3 29. ? @ 0x00007f12cd7f0850 : While executing Remote. (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::readException(DB::ReadBuffer&, String const&, bool) @ 0x000000001aeb974e 3. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::Connection::receiveException() const @ 0x000000002cee6c6a 4. ./build_docker/./src/Client/Connection.cpp:0: DB::Connection::receivePacket() @ 0x000000002cef780e 5. ./build_docker/./src/Client/MultiplexedConnections.cpp:340: DB::MultiplexedConnections::receivePacketUnlocked(std::function) @ 0x000000002cf8e09e 6. ./build_docker/./src/QueryPipeline/RemoteQueryExecutorReadContext.cpp:0: DB::RemoteQueryExecutorReadContext::Task::run(std::function, std::function) @ 0x00000000278ad616 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278ab436 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000278aaa44 9. ./contrib/boost/boost/context/fiber_ucontext.hpp:82: void boost::context::detail::fiber_entry_func>>(unsigned int, unsigned int) @ 0x00000000278a8394 2025.09.25 21:26:56.943078 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:57.204091 [ 3029 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:57.236661 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:57.450438 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:57.757531 [ 3047 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:57.758773 [ 3048 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:57.953742 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:57.990571 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:58.186464 [ 3350 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {261a6bdf-2436-455f-b6df-dbd9b591b8be::202509_1_174_57}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:58.205930 [ 3068 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:58.250644 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:58.674916 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:58.732189 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:58.760964 [ 3091 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:58.761985 [ 3084 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:58.819917 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:59.233558 [ 3106 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:59.269422 [ 10192 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:59.490851 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:59.554079 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:59.567972 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:59.763872 [ 3112 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:26:59.764858 [ 3108 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:00.236772 [ 3120 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:00.374516 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:00.388033 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:00.526068 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:00.769104 [ 3138 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:00.769178 [ 3151 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:01.125209 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:01.152703 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:01.239841 [ 3140 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:01.285219 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:01.773346 [ 3153 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:01.773883 [ 3166 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:01.838547 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:01.855519 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:01.978007 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:02.243071 [ 3175 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:02.253042 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:02.736897 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:02.776858 [ 3187 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:02.776982 [ 3097 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:02.787402 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:02.826175 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:03.019303 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:03.149686 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:03.245510 [ 3169 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:03.459819 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:03.521113 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:03.779514 [ 3204 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:03.779791 [ 3216 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:03.955648 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:04.247700 [ 3215 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:04.340096 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:04.373889 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:04.385379 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:04.557517 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:04.772843 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:04.781982 [ 3239 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:04.868115 [ 3250 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:04.868239 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:05.172739 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:05.249798 [ 2756 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:05.292479 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:05.316817 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:05.765308 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:05.783972 [ 2766 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:05.793132 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:05.870193 [ 2775 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:05.981708 [ 13110 ] {0e640376-eb01-47c5-9cdb-5aaa3c4c8e60} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=2). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:35630) (comment: 02844_max_backup_bandwidth_s3.sh) (in query: -- reading 1e6*8 bytes with 1M bandwith it should take (8-1)/1=7 seconds insert into data select * from numbers(1e6);), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002699bf33 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000002699ba34 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x0000000026b1a342 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000026aeba41 10. ./contrib/llvm-project/libcxx/include/__split_buffer:88: DB::WriteBufferFromS3::preFinalize() @ 0x0000000026aeaa5e 11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:69: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x00000000280d209a 12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr const&, DB::MergeTreeDataPartChecksums&) @ 0x000000002c8e2519 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*, std::vector>*) @ 0x000000002c8df1f4 14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:683: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr const&, std::shared_ptr, long, bool) @ 0x000000002c95e692 15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:444: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr const&, std::shared_ptr) @ 0x000000002c958ddf 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk&) @ 0x000000002cc2a689 17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:11: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000002dbe182e 18. ./contrib/llvm-project/libcxx/include/vector:432: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d9c1a3a 19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function, DB::ThreadStatus*, std::atomic*) @ 0x000000002d9c1551 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d9c06a6 21. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 25. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007f12cd75eac3 30. ? @ 0x00007f12cd7f0850 2025.09.25 21:27:05.991322 [ 13110 ] {0e640376-eb01-47c5-9cdb-5aaa3c4c8e60} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=2). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002699bf33 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x000000002699ba34 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x0000000026b1a342 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x0000000026aeba41 10. ./contrib/llvm-project/libcxx/include/__split_buffer:88: DB::WriteBufferFromS3::preFinalize() @ 0x0000000026aeaa5e 11. ./build_docker/./src/IO/WriteBufferFromFileDecorator.cpp:69: DB::WriteBufferFromFileDecorator::preFinalize() @ 0x00000000280d209a 12. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:0: DB::MergedBlockOutputStream::finalizePartOnDisk(std::shared_ptr const&, DB::MergeTreeDataPartChecksums&) @ 0x000000002c8e2519 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::MergedBlockOutputStream::finalizePartAsync(std::shared_ptr const&, bool, DB::NamesAndTypesList const*, DB::MergeTreeDataPartChecksums*, std::vector>*) @ 0x000000002c8df1f4 14. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:683: DB::MergeTreeDataWriter::writeTempPartImpl(DB::BlockWithPartition&, std::shared_ptr const&, std::shared_ptr, long, bool) @ 0x000000002c95e692 15. ./build_docker/./src/Storages/MergeTree/MergeTreeDataWriter.cpp:444: DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr const&, std::shared_ptr) @ 0x000000002c958ddf 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSink.cpp:0: DB::MergeTreeSink::consume(DB::Chunk&) @ 0x000000002cc2a689 17. ./build_docker/./src/Processors/Sinks/SinkToStorage.cpp:11: DB::SinkToStorage::onConsume(DB::Chunk) @ 0x000000002dbe182e 18. ./contrib/llvm-project/libcxx/include/vector:432: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d9c1a3a 19. ./build_docker/./src/Processors/Transforms/ExceptionKeepingTransform.cpp:102: DB::runStep(std::function, DB::ThreadStatus*, std::atomic*) @ 0x000000002d9c1551 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d9c06a6 21. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 22. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 23. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 25. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 28. asan_thread_start(void*) @ 0x000000000aa43059 29. ? @ 0x00007f12cd75eac3 30. ? @ 0x00007f12cd7f0850 2025.09.25 21:27:06.161686 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:06.252114 [ 2789 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:06.687715 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:06.723174 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:06.786913 [ 2787 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:06.873575 [ 2795 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:07.220219 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:07.239400 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:07.253837 [ 2815 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:07.339164 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:07.789246 [ 2822 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:07.876538 [ 2834 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:08.260148 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:08.262171 [ 2850 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:08.406236 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:08.509865 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:08.791966 [ 2859 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:08.879332 [ 2867 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:08.895331 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:09.265468 [ 2876 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:09.409451 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:09.459958 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:09.799382 [ 2901 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:09.882462 [ 2891 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:09.998660 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:10.242580 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:10.268398 [ 2910 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:10.442487 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:10.638746 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:10.801772 [ 2924 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:10.885807 [ 2923 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:11.291399 [ 2940 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:11.303582 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:11.414582 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:11.519809 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:11.750736 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:11.805207 [ 2954 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:11.889143 [ 2949 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:12.294722 [ 2965 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:12.314263 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:12.775405 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:12.810526 [ 2992 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:12.849849 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:13.005502 [ 2994 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:13.300946 [ 3008 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:13.310144 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:13.460545 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:13.706828 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:13.813496 [ 3014 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:13.903919 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:13.939135 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:14.008053 [ 3028 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:14.303671 [ 3041 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:14.457016 [ 13110 ] {2ddf5cdf-53ab-44c0-8c85-0b1e52577b56} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=0): (while reading column query_duration_ms): (while reading from part /var/lib/clickhouse/store/f81/f81dbc1c-a1bd-4968-9514-f0c97137169a/202509_158_158_0/ in table system.query_log (f81dbc1c-a1bd-4968-9514-f0c97137169a) located on disk default of type local, from mark 0 with max_rows_to_read = 10): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:44618) (comment: 02844_max_backup_bandwidth_s3.sh) (in query: SELECT 'no_native_copy', query_duration_ms >= 7e3 FROM system.query_log WHERE current_database = 'test_a6aed4q0' AND query_id = 'lkvletuldt' AND type != 'QueryStart' ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. ./src/IO/ReadBuffer.h:0: DB::CompressedReadBufferBase::readCompressedDataBlockForAsynchronous(unsigned long&, unsigned long&) @ 0x0000000027760e7f 13. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:110: DB::CompressedReadBufferFromFile::readBig(char*, unsigned long) @ 0x0000000027764c41 14. ./build_docker/./src/DataTypes/Serializations/SerializationNumber.cpp:222: DB::SerializationNumber::deserializeBinaryBulk(DB::IColumn&, DB::ReadBuffer&, unsigned long, double) const @ 0x0000000027b97730 15. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027aa00db 16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::readData(unsigned long, COW::immutable_ptr&, unsigned long, DB::MergeTreeReaderStream&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) @ 0x000000002c7b254b 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.cpp:53: DB::MergeTreeReaderCompactSingleBuffer::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7eb79e 18. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 19. ./src/Storages/MergeTree/MergeTreeRangeReader.h:139: DB::MergeTreeRangeReader::continueReadingChain(DB::MergeTreeRangeReader::ReadResult const&, unsigned long&) @ 0x000000002c7df38d 20. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:984: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbd9d 21. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 22. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:38: DB::MergeTreeThreadSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002de66094 23. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 24. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 25. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 26. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 27. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 28. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 30. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 31. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 2025.09.25 21:27:14.467729 [ 13110 ] {2ddf5cdf-53ab-44c0-8c85-0b1e52577b56} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=5, jobs=0): (while reading column query_duration_ms): (while reading from part /var/lib/clickhouse/store/f81/f81dbc1c-a1bd-4968-9514-f0c97137169a/202509_158_158_0/ in table system.query_log (f81dbc1c-a1bd-4968-9514-f0c97137169a) located on disk default of type local, from mark 0 with max_rows_to_read = 10): While executing MergeTreeSelect(pool: ReadPool, algorithm: Thread). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. ./src/IO/ReadBuffer.h:0: DB::CompressedReadBufferBase::readCompressedDataBlockForAsynchronous(unsigned long&, unsigned long&) @ 0x0000000027760e7f 13. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:110: DB::CompressedReadBufferFromFile::readBig(char*, unsigned long) @ 0x0000000027764c41 14. ./build_docker/./src/DataTypes/Serializations/SerializationNumber.cpp:222: DB::SerializationNumber::deserializeBinaryBulk(DB::IColumn&, DB::ReadBuffer&, unsigned long, double) const @ 0x0000000027b97730 15. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:0: DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW::immutable_ptr&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) const @ 0x0000000027aa00db 16. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::readData(unsigned long, COW::immutable_ptr&, unsigned long, DB::MergeTreeReaderStream&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>&, std::unordered_map::immutable_ptr, std::hash, std::equal_to, std::allocator::immutable_ptr>>>*) @ 0x000000002c7b254b 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.cpp:53: DB::MergeTreeReaderCompactSingleBuffer::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7eb79e 18. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 19. ./src/Storages/MergeTree/MergeTreeRangeReader.h:139: DB::MergeTreeRangeReader::continueReadingChain(DB::MergeTreeRangeReader::ReadResult const&, unsigned long&) @ 0x000000002c7df38d 20. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:984: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbd9d 21. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 22. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:38: DB::MergeTreeThreadSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002de66094 23. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 24. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 25. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 26. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 27. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 28. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:381: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000002d34bd35 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 30. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 31. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 2025.09.25 21:27:14.705658 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:14.738342 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:14.815370 [ 3055 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:14.826661 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:15.010080 [ 3056 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:15.309849 [ 3066 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:15.322039 [ 13129 ] {} TCPHandler: Code: 452. DB::Exception: Setting allow_experimental_analyzer should not be changed. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception>&>(int, FormatStringHelperImpl>&>::type>, std::basic_string_view>&) @ 0x0000000011c286b4 4. ./build_docker/./src/Access/SettingsConstraints.cpp:332: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f8148e 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/remove_if.h:32: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f80932 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./build_docker/./src/Server/TCPHandler.cpp:2034: DB::TCPHandler::receiveQuery() @ 0x000000002d1a17ad 10. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::receivePacket() @ 0x000000002d1877e2 11. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d175061 12. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:27:15.377524 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:15.536142 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:15.706707 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:15.818205 [ 3088 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:15.913833 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:15.955847 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:16.011719 [ 3081 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:16.312624 [ 3095 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:16.360082 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:16.873061 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:16.919724 [ 3111 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:16.996106 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:17.014267 [ 3108 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:17.026646 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:17.314824 [ 3131 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:17.706367 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:17.932869 [ 3136 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:17.967697 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:18.002212 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:18.017360 [ 3140 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:18.318491 [ 3142 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:18.403671 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:18.435202 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:18.813325 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:18.936161 [ 3171 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:19.020362 [ 3178 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:19.086251 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:19.192230 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:19.321739 [ 3183 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:19.468796 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:19.489210 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:19.793849 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:19.926867 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:19.935994 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:19.939297 [ 3200 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:20.007439 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:20.023577 [ 3193 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:20.322968 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:20.324633 [ 3223 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:20.446943 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:20.557327 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:20.617494 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:20.722232 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:20.951202 [ 3238 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:20.974523 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:21.025568 [ 3239 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:21.327526 [ 3234 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:21.597084 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:21.660112 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:21.681575 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:21.897685 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:21.953867 [ 2763 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:22.028476 [ 2780 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:22.135527 [ 13129 ] {4776377c-dcb7-4c32-bec4-50b93ee66049} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=735). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48774) (comment: 01475_read_subcolumns_3.sql) (in query: SELECT count() FROM map_subcolumns WHERE arrayMax(m.values) > 3;), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:27:22.136518 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:22.140253 [ 13129 ] {4776377c-dcb7-4c32-bec4-50b93ee66049} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=735). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:27:22.179145 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:22.322958 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:22.387800 [ 2788 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:22.496177 [ 13110 ] {9345e8e9-c658-49f0-b7c9-a11af6b17abe} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48816) (comment: 01475_read_subcolumns_3.sql) (in query: DROP DATABASE IF EXISTS test_7fbb3aq9 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:27:22.499513 [ 13110 ] {9345e8e9-c658-49f0-b7c9-a11af6b17abe} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:22.598157 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:22.956245 [ 2796 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:22.975681 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:22.992992 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:23.034454 [ 2812 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:23.116712 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:23.446506 [ 2808 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:23.521835 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:23.834101 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:23.960375 [ 2834 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:24.038798 [ 2839 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:24.156745 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:24.179876 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:24.449881 [ 2840 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:24.841716 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:24.940697 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:24.964068 [ 2854 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:25.042610 [ 2858 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:25.067106 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:25.452899 [ 2859 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:25.492048 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:25.631163 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:25.729742 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:25.966995 [ 2885 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:26.046501 [ 2890 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:26.231391 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:26.456454 [ 2893 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:26.555561 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:26.670770 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:26.970310 [ 2903 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:27.049751 [ 2900 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:27.073794 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:27.443207 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:27.459853 [ 2924 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:27.551693 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:27.608858 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:27.973523 [ 2947 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:28.062983 [ 2937 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:28.132927 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:28.179112 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:28.304989 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:28.467233 [ 2955 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:28.562436 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:28.587699 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:28.688359 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:28.976763 [ 2967 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:29.066467 [ 2987 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:29.301775 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:29.470177 [ 2983 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:29.486095 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:29.583158 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:29.788325 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:29.980509 [ 2993 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:30.069473 [ 2998 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:30.472351 [ 3017 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:30.671458 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:30.891806 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:30.987988 [ 3024 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:30.998856 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:31.072534 [ 3016 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:31.477755 [ 3028 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:31.525970 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:31.651468 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:31.671347 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:31.991065 [ 3049 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:32.075649 [ 3046 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:32.187152 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:32.205282 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:32.205926 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:32.356883 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:32.481229 [ 3061 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:32.601674 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:32.713392 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:32.994205 [ 3068 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:33.078783 [ 3073 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:33.151028 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:33.283188 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:33.313170 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:33.484602 [ 3099 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:33.953528 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:33.996567 [ 3107 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:34.081000 [ 3098 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:34.246039 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:34.270831 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:34.486775 [ 3118 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:34.760333 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:34.776300 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:34.999663 [ 3120 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:35.070551 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:35.084216 [ 3124 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:35.412718 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:35.452643 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:35.497453 [ 3150 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:35.566825 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:36.005746 [ 3142 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:36.087769 [ 3153 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:36.209216 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:36.246846 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:36.485781 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:36.508416 [ 3168 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:37.008821 [ 3182 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:37.091079 [ 3188 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:37.349549 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:37.396116 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:37.413721 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:37.512025 [ 3179 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:37.766044 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:38.019054 [ 3204 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:38.094329 [ 3213 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:38.289952 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:38.364722 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:38.389846 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:38.515302 [ 3231 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:39.056992 [ 3246 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:39.097163 [ 3247 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:39.357786 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:39.466511 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:39.497616 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:39.518441 [ 3260 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:40.059063 [ 3254 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:40.100418 [ 2761 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:40.328827 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:40.359772 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:40.394458 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:40.521276 [ 2773 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:40.806959 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:41.103942 [ 2771 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:41.062073 [ 2770 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:41.530943 [ 2750 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:41.742860 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:41.774732 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:41.840129 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:42.106926 [ 2817 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:42.111688 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:42.115725 [ 2820 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:42.366373 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:42.421842 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:42.538966 [ 2818 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:42.697059 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:42.961569 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:42.989002 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:43.109693 [ 2838 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:43.118842 [ 2848 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:43.541947 [ 2858 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:43.748548 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:43.950309 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:44.057039 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:44.113613 [ 2867 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:44.123866 [ 2869 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:44.150214 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:44.545202 [ 2885 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:44.634573 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:44.770211 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:44.796733 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:45.116960 [ 2906 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:45.127162 [ 2904 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:45.524580 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:45.551761 [ 2915 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:45.842463 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:45.896766 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:45.974516 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:46.119208 [ 2923 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:46.130527 [ 2918 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:46.378983 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:46.390488 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:46.555927 [ 2946 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:46.593414 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:46.969271 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:46.987937 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:47.121221 [ 2948 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:47.132602 [ 2955 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:47.526737 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:47.558163 [ 2969 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:47.862635 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:47.874393 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:47.915599 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:48.124245 [ 2979 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:48.135473 [ 2972 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:48.561105 [ 2977 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:48.778353 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:48.791024 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:48.824536 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:48.995185 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:49.127419 [ 3017 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:49.138220 [ 3002 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:49.563199 [ 3010 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:49.942054 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:49.952026 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:49.961557 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:50.129638 [ 3033 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:50.142687 [ 3023 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:50.565447 [ 3041 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:50.667253 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:50.873805 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:50.900585 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:51.131873 [ 3058 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:51.144874 [ 3063 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:51.424528 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:51.567476 [ 3080 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:51.609034 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:51.618965 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:51.866686 [ 13110 ] {91f4848e-0d18-4397-a5ac-0863259cb45e} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:47554) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:27:51.871543 [ 13110 ] {91f4848e-0d18-4397-a5ac-0863259cb45e} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:27:52.136801 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:52.139257 [ 3088 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:52.147083 [ 3079 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:52.354991 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:52.369415 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:52.524856 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:52.569832 [ 3100 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:52.778754 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:52.789581 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:53.141214 [ 3110 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:53.149124 [ 3112 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:53.401127 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:53.411211 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:53.556070 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:53.572621 [ 3124 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:53.952049 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:53.996666 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:54.143179 [ 3130 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:54.150994 [ 3148 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:54.220271 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:54.234758 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:54.428611 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:54.476186 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:54.493580 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:54.575397 [ 3162 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:54.977149 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:55.122349 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:55.144516 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:55.144817 [ 3171 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:55.152736 [ 3177 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:55.602795 [ 3170 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:55.782998 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:55.823734 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:55.990485 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:56.147767 [ 3197 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:56.155286 [ 3191 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:56.605953 [ 3200 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:56.940645 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:56.950089 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:56.964918 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:57.149831 [ 3217 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:57.157325 [ 3216 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:57.609595 [ 3221 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:57.848164 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:57.976753 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:57.995202 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:58.152349 [ 3240 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:58.159921 [ 3227 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:58.535755 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:58.614956 [ 3255 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:58.711089 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:58.746044 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:58.968552 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:59.155622 [ 2764 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:59.163098 [ 3259 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:59.265668 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:59.290815 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:59.618185 [ 2769 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:59.751518 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:27:59.948572 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:00.158337 [ 2765 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:00.165875 [ 2785 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:00.220096 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:00.484850 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:00.516183 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:00.621752 [ 2799 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:01.091518 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:01.165830 [ 2813 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:01.168133 [ 2805 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:01.287923 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:01.305480 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:01.599466 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:01.634141 [ 2830 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:01.894911 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:02.168403 [ 2838 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:02.254051 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:02.266825 [ 2848 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:02.285677 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:02.637287 [ 2853 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:02.817410 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:02.955710 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:03.039330 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:03.171135 [ 2865 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:03.270412 [ 2867 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:03.457730 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:03.505682 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:03.641018 [ 2885 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:03.692330 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:04.157387 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:04.178175 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:04.179034 [ 2837 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:04.273341 [ 2901 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:04.523719 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:04.644230 [ 2905 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:05.182024 [ 2913 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:05.187922 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:05.227300 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:05.274072 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:05.276124 [ 2917 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:05.647133 [ 2920 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:05.821250 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:06.187298 [ 2939 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:06.279103 [ 2945 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:06.311334 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:06.391440 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:06.412923 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:06.606833 [ 3350 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {fffcce7c-f4e8-42ed-b1c9-22aa3ca82e93::202509_187_202_3}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:06.653357 [ 2962 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:07.042431 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:07.190328 [ 2975 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:07.281240 [ 2979 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:07.295452 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:07.330083 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:07.345040 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:07.656478 [ 2994 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:08.160268 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:08.194650 [ 3002 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:08.201671 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:08.228911 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:08.283788 [ 3010 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:08.587253 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:08.673321 [ 3026 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:08.688969 [ 3362 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {f81dbc1c-a1bd-4968-9514-f0c97137169a::202509_1_160_37}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001adfc17a 4. ./build_docker/./src/Common/MemoryTracker.cpp:187: MemoryTracker::injectFault() const @ 0x000000001adf8d1a 5. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x000000002cc30a65 6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f47b 7. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:09.012877 [ 13110 ] {5f255dc2-4ba2-4249-8a2c-690e1fd266d7} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=4): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53516) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027b55cdf 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:28:09.018072 [ 13110 ] {5f255dc2-4ba2-4249-8a2c-690e1fd266d7} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=4): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027b55cdf 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:28:09.072622 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:09.097265 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:09.199087 [ 3028 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:09.286806 [ 3034 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:09.363218 [ 13110 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:09.423201 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:09.676655 [ 3049 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:09.981888 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:10.206504 [ 3063 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:10.244503 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:10.285715 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:10.289871 [ 3054 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:10.654851 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:10.678817 [ 3075 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:11.085700 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:11.124350 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:11.209886 [ 3088 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:11.297372 [ 3091 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:11.664692 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:11.683628 [ 3082 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:12.040473 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:12.057037 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:12.222799 [ 3077 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:12.299144 [ 3110 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:12.313520 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:12.321492 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:12.460764 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:12.685970 [ 3115 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:13.226228 [ 3126 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:13.301734 [ 3128 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:13.319228 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:13.346092 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:13.506452 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:13.688606 [ 3145 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:14.057857 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:14.189555 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:14.205170 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:14.229380 [ 3171 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:14.304842 [ 3173 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:14.453140 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:14.654058 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:14.682996 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:14.691811 [ 3181 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:14.886896 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:15.134389 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:15.232348 [ 3194 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:15.309376 [ 3199 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:15.574048 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:15.639824 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:15.766131 [ 3211 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:16.159831 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:16.234222 [ 3219 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:16.317192 [ 3212 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:16.380273 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:16.671320 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:16.683339 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:16.768802 [ 3233 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:17.068784 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:17.236847 [ 3235 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:17.320096 [ 3256 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:17.550535 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:17.576660 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:17.771810 [ 2752 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:17.894941 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:18.115392 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:18.117791 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:18.142412 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:18.243940 [ 2776 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:18.323192 [ 2780 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:18.496818 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:18.775071 [ 2789 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:19.030562 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:19.096093 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:19.247087 [ 2758 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:19.287442 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:19.326362 [ 2782 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:19.778055 [ 2799 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:19.901178 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:19.921688 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:20.107302 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:20.250050 [ 2805 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:20.331846 [ 2806 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:20.400322 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:20.415150 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:20.781386 [ 2818 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:20.807611 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:20.994917 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:21.253377 [ 2835 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:21.338443 [ 2841 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:21.473486 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:21.605006 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:21.664814 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:21.784490 [ 2854 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:21.847063 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:22.256456 [ 2866 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:22.341649 [ 2872 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:22.583553 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:22.596614 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:22.696600 [ 13129 ] {3f3c99e8-da05-4873-885d-529680c2385e} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48818) (comment: 01475_read_subcolumns_3.sql) (in query: DROP DATABASE IF EXISTS test_7fbb3aq9 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:28:22.700009 [ 13129 ] {3f3c99e8-da05-4873-885d-529680c2385e} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:22.828574 [ 13110 ] {b32b5e2c-0f63-4d1f-b4c1-d8edfb20e7ad} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:52250) (comment: 01475_read_subcolumns_3.sql) (in query: DROP DATABASE IF EXISTS test_7fbb3aq9 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:28:22.834393 [ 13110 ] {b32b5e2c-0f63-4d1f-b4c1-d8edfb20e7ad} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:22.851077 [ 2884 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:22.886441 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:22.899134 [ 3816 ] {65e6b6f6-db4c-4045-ba07-2e8783e32b6e} executeQuery: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:52258) (comment: 01475_read_subcolumns_3.sql) (in query: DROP DATABASE IF EXISTS test_7fbb3aq9 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 2025.09.25 21:28:22.904019 [ 3816 ] {65e6b6f6-db4c-4045-ba07-2e8783e32b6e} DynamicQueryHandler: Code: 219. DB::Exception: New table appeared in database being dropped or detached. Try again. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000aac868e 4. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:591: DB::DatabaseCatalog::detachDatabase(std::shared_ptr, String const&, bool, bool) @ 0x00000000286b57c7 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr&, std::vector, DB::UUIDTag>, std::allocator, DB::UUIDTag>>>&) @ 0x00000000291f5630 6. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:361: DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x00000000291edc97 7. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:0: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291ebf11 8. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291eb7ed 9. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 11. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 12. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 13. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 17. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:23.263443 [ 2886 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:23.344782 [ 2895 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:23.455781 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:23.539892 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:23.853717 [ 2915 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:23.871609 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:24.267187 [ 2926 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:24.347832 [ 2911 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:24.644942 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:24.786297 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:24.817230 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:24.856820 [ 2941 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:25.098892 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:25.126610 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:25.270044 [ 2937 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:25.429378 [ 2949 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:25.458474 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:25.472749 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:25.580506 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:25.860261 [ 2963 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:26.105637 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:26.180629 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:26.273144 [ 2969 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:26.406328 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:26.432624 [ 2976 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:26.568629 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:26.863621 [ 2980 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:27.241290 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:27.266307 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:27.275978 [ 2997 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:27.436350 [ 3005 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:27.655710 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:27.867288 [ 3002 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:28.278826 [ 3020 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:28.338161 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:28.367525 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:28.420882 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:28.446921 [ 3016 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:28.598550 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:28.869944 [ 3030 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:28.955291 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:28.975314 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:29.260978 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:29.281220 [ 3046 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:29.434910 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:29.449943 [ 3050 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:29.453860 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:29.783531 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:29.872640 [ 3061 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:30.124010 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:30.149823 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:30.243612 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:30.283334 [ 3073 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:30.452496 [ 3071 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:30.629420 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:30.874834 [ 3090 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:30.953033 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:30.972911 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:31.285526 [ 3105 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:31.455158 [ 3106 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:31.761857 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:31.877571 [ 3113 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:32.056824 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:32.076683 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:32.197918 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:32.288539 [ 3124 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:32.458007 [ 3135 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:32.563113 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:32.593531 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:32.639521 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:32.880442 [ 3139 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:33.051621 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:33.291890 [ 3149 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:33.460850 [ 3142 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:33.829469 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:33.883159 [ 3166 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:33.910729 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:34.155251 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:34.295421 [ 3168 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:34.464314 [ 3183 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:34.885007 [ 3196 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:34.915532 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:35.228437 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:35.238272 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:35.298049 [ 3194 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:35.466882 [ 3202 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:35.891764 [ 3169 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:36.014707 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:36.215714 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:36.300150 [ 3208 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:36.405261 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:36.469310 [ 3223 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:36.843267 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:36.893989 [ 3240 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:36.995356 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:37.021884 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:37.309564 [ 3230 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:37.356615 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:37.471978 [ 3260 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:37.496542 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:37.510855 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:37.890283 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:37.896065 [ 3259 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:38.200482 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:38.218068 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:38.312520 [ 2780 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:38.474453 [ 2781 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:38.615850 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:38.635733 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:38.856622 [ 3816 ] {0ec7ad55-c95b-4771-8e76-80fbfbf16367} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:44386) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:28:38.860363 [ 3816 ] {0ec7ad55-c95b-4771-8e76-80fbfbf16367} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:28:38.898623 [ 2783 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:38.994385 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:39.314855 [ 2750 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:39.476917 [ 2797 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:39.512817 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:39.817816 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:39.835985 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:39.900833 [ 2815 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:40.090306 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:40.317003 [ 2808 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:40.479070 [ 2809 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:40.555895 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:40.902725 [ 2834 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:40.988732 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:41.001751 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:41.011503 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:41.319099 [ 2841 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:41.486654 [ 2845 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:41.613051 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:41.890281 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:41.904660 [ 2854 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:41.929108 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:42.321212 [ 2847 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:42.488777 [ 2866 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:42.494774 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:42.835914 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:42.853704 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:42.906647 [ 2874 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:43.064613 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:43.323920 [ 2882 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:43.491579 [ 2877 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:43.908553 [ 2902 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:43.913988 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:43.950960 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:43.990992 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:44.326219 [ 2894 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:44.494469 [ 2911 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:44.889181 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:44.912578 [ 2925 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:45.260542 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:45.290269 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:45.328932 [ 2826 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:45.497166 [ 2938 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:45.588658 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:45.915743 [ 2948 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:46.028663 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:46.295467 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:46.331969 [ 2987 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:46.388806 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:46.431955 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:46.500641 [ 2968 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:46.919418 [ 2981 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:47.046979 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:47.095087 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:47.124218 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:47.336038 [ 3000 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:47.510057 [ 3008 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:47.620614 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:47.922521 [ 3020 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:48.016728 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:48.054947 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:48.339647 [ 3033 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:48.446489 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:48.467767 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:48.515652 [ 3034 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:48.516321 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:48.594689 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:48.977713 [ 3049 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:49.010659 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:49.049109 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:49.342802 [ 3052 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:49.373475 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:49.430943 [ 3816 ] {a8435dc2-4838-4a07-b690-60eac35a7d56} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37092) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:28:49.435173 [ 3816 ] {a8435dc2-4838-4a07-b690-60eac35a7d56} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:28:49.502876 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:49.519642 [ 3059 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:49.980707 [ 3080 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:50.048370 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:50.069734 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:50.345433 [ 3072 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:50.489733 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:50.522461 [ 3071 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:50.920520 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:50.936323 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:50.983572 [ 3081 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:51.313593 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:51.347805 [ 3092 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:51.525017 [ 3103 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:51.583927 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:51.692625 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:51.986541 [ 3112 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:52.081668 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:52.117240 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:52.433363 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:52.444547 [ 3134 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:52.528293 [ 3126 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:52.615157 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:52.728418 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:52.746285 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:52.988827 [ 3146 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:53.446922 [ 3162 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:53.531040 [ 3153 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:53.576039 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:53.881502 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:53.906592 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:53.991393 [ 3168 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:54.313376 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:54.449698 [ 3182 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:54.533815 [ 3187 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:54.994285 [ 3202 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:55.206607 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:55.209644 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:55.235350 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:55.452227 [ 3209 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:55.536702 [ 3216 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:55.996731 [ 3212 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:56.096164 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:56.429717 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:56.447250 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:56.454845 [ 3241 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:56.539465 [ 3243 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:56.824635 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:56.855025 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:56.923040 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:56.999735 [ 3248 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:57.421149 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:57.458335 [ 3257 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:57.542222 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:57.542747 [ 3254 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:57.557224 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:57.570053 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:58.006590 [ 2779 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:58.019011 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:58.046267 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:58.460826 [ 2769 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:58.545653 [ 2755 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:58.578006 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:58.988640 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:59.009690 [ 2786 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:59.262309 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:59.286241 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:59.463652 [ 2791 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:28:59.551611 [ 2799 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:00.012766 [ 2820 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:00.089000 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:00.120536 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:00.156197 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:00.467094 [ 2822 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:00.534754 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:00.556489 [ 2836 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:00.825590 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:01.018361 [ 2823 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:01.113718 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:01.126248 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:01.371613 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:01.470545 [ 2864 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:01.559892 [ 2859 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:01.705584 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:02.021329 [ 2879 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:02.043822 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:02.095466 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:02.473630 [ 2837 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:02.563063 [ 2891 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:02.573763 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:02.970055 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:02.986797 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:03.025330 [ 2899 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:03.366341 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:03.420356 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:03.477183 [ 2921 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:03.571087 [ 2914 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:03.722732 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:03.951213 [ 3816 ] {aa31c3b4-d05a-4240-a05e-8195c11cab8c} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:37244) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:29:03.956528 [ 3816 ] {aa31c3b4-d05a-4240-a05e-8195c11cab8c} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:29:04.028832 [ 2927 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:04.377675 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:04.394530 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:04.481105 [ 2826 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:04.574466 [ 2947 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:04.817086 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:05.032357 [ 2958 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:05.271176 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:05.307486 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:05.461339 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:05.484012 [ 2960 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:05.577719 [ 2974 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:06.035461 [ 2985 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:06.142153 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:06.164314 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:06.487308 [ 2994 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:06.581179 [ 2993 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:06.619893 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:07.038231 [ 3008 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:07.409455 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:07.463197 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:07.490830 [ 3033 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:07.578319 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:07.583634 [ 3027 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:08.041881 [ 3042 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:08.236161 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:08.494213 [ 3047 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:08.586932 [ 3050 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:08.773247 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:08.790392 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:09.045200 [ 3057 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:09.204974 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:09.410685 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:09.441989 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:09.497525 [ 3073 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:09.590000 [ 3071 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:10.047785 [ 3095 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:10.050404 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:10.191782 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:10.218741 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:10.500526 [ 3101 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:10.592366 [ 3104 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:10.705775 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:10.718966 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:10.739812 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:10.915099 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:11.050819 [ 3131 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:11.505966 [ 3135 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:11.571624 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:11.595160 [ 3130 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:11.960440 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:12.043143 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:12.053932 [ 3145 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:12.383526 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:12.509226 [ 3153 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:12.600357 [ 3164 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:12.771982 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:12.818087 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:12.881458 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:13.057642 [ 3173 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:13.259783 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:13.276059 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:13.348908 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:13.535620 [ 3180 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:13.550628 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:13.574038 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:13.602641 [ 3197 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:13.703639 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:14.061061 [ 3210 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:14.258574 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:14.277989 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:14.539162 [ 3209 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:14.605535 [ 3216 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:14.717273 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:15.064415 [ 3218 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:15.364439 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:15.542438 [ 3249 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:15.579946 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:15.604153 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:15.610865 [ 3247 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:16.067228 [ 2745 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:16.177297 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:16.545576 [ 2754 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:16.615483 [ 2768 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:16.647782 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:16.680615 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:17.069829 [ 2769 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:17.235551 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:17.548956 [ 2783 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:17.618815 [ 2788 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:17.786394 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:17.828221 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:18.085468 [ 2784 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:18.225540 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:18.507634 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:18.554930 [ 2817 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:18.556323 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:18.622690 [ 2827 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:19.057596 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:19.088785 [ 2809 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:19.355442 [ 3816 ] {01820383-432d-45f9-ac8a-06de7709c3bc} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50348) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:29:19.359892 [ 3816 ] {01820383-432d-45f9-ac8a-06de7709c3bc} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:29:19.420813 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:19.557592 [ 2831 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:19.625435 [ 2836 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:19.771543 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:19.939996 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:20.057774 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:20.102906 [ 2856 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:20.174706 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:20.560920 [ 2860 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:20.621145 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:20.628370 [ 2864 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:20.816578 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:20.827728 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:21.105315 [ 2870 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:21.257871 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:21.563682 [ 2884 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:21.665654 [ 2880 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:21.677763 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:21.831871 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:22.073216 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:22.088777 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:22.107413 [ 2902 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:22.111879 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:22.512154 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:22.529802 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:22.557931 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:22.566266 [ 2922 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:22.668330 [ 2924 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:23.110323 [ 2925 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:23.113539 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:23.212352 [ 3816 ] {47df5e5d-0546-4d00-ab87-905c50284a0a} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50374) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:29:23.221020 [ 3816 ] {47df5e5d-0546-4d00-ab87-905c50284a0a} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:29:23.513432 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:23.531108 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:23.569418 [ 2930 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:23.755749 [ 2947 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:24.112809 [ 2958 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:24.194816 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:24.232962 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:24.250819 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:24.543603 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:24.572204 [ 2974 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:24.744515 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:24.758340 [ 2978 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:25.115832 [ 2970 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:25.167276 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:25.215327 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:25.361938 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:25.584032 [ 2991 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:25.604491 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:25.635933 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:25.760449 [ 2996 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:26.118723 [ 3015 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:26.263716 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:26.587797 [ 3022 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:26.765054 [ 3016 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:26.840312 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:26.869445 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:27.090977 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:27.108876 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:27.130538 [ 3036 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:27.244124 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:27.590098 [ 3049 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:27.712352 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:27.767724 [ 3053 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:27.909766 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:27.927994 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:28.133122 [ 3059 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:28.397164 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:28.399246 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:28.419389 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:28.593000 [ 3076 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:28.770155 [ 3068 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:28.952892 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:29.081540 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:29.108552 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:29.134886 [ 3086 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:29.233930 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:29.366608 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:29.377976 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:29.439682 [ 13110 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:29.598403 [ 3102 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:29.772283 [ 3106 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:30.137679 [ 3112 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:30.187295 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:30.204998 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:30.251249 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:30.432326 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:30.471338 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:30.601272 [ 3134 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:30.775930 [ 3133 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:30.851365 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:30.892048 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:31.141363 [ 3146 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:31.397096 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:31.608953 [ 3164 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:31.673200 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:31.718774 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:31.780008 [ 3160 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:32.016171 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:32.144318 [ 3175 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:32.450728 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:32.505395 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:32.611889 [ 3191 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:32.791625 [ 3198 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:32.845749 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:32.881487 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:32.883035 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:33.051863 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:33.147588 [ 3192 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:33.449100 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:33.614595 [ 3215 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:33.794470 [ 3218 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:33.879105 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:33.909496 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:34.067467 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:34.150986 [ 3241 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:34.641469 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:34.667702 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:34.691501 [ 3256 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:34.797950 [ 3230 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:35.118880 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:35.159089 [ 3258 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:35.456645 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:35.647333 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:35.694979 [ 2762 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:35.737815 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:35.801227 [ 3259 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:35.878027 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:36.161725 [ 2780 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:36.488734 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:36.636862 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:36.697729 [ 2785 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:36.715400 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:36.743265 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:36.804082 [ 2758 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:37.164949 [ 2787 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:37.699675 [ 2813 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:37.760898 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:37.806349 [ 2820 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:37.880001 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:37.889342 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:38.185191 [ 2830 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:38.421200 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:38.708434 [ 2835 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:38.808992 [ 2823 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:39.010898 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:39.031810 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:39.188310 [ 2849 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:39.285871 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:39.463868 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:39.506184 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:39.711292 [ 2866 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:39.812715 [ 2868 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:39.846317 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:39.955118 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:39.986854 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:40.190719 [ 2890 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:40.360260 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:40.714186 [ 2882 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:40.815635 [ 2893 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:40.830308 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:40.923532 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:40.963350 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:41.192554 [ 2899 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:41.717895 [ 2923 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:41.758426 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:41.818757 [ 2918 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:41.913006 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:41.931901 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:42.195208 [ 2934 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:42.417740 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:42.436475 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:42.758122 [ 2945 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:42.821010 [ 2954 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:42.825497 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:43.077647 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:43.101057 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:43.219806 [ 2952 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:43.513995 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:43.761313 [ 2960 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:43.823218 [ 2974 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:43.941373 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:44.029495 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:44.223017 [ 2970 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:44.253305 [ 3816 ] {d22a081c-e50f-4ac3-8759-171c417f9314} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:40522) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:29:44.258656 [ 3816 ] {d22a081c-e50f-4ac3-8759-171c417f9314} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:29:44.470277 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:44.764708 [ 2982 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:44.826027 [ 2990 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:45.140158 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:45.226723 [ 3015 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:45.250260 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:45.283871 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:45.733024 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:45.767412 [ 3016 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:45.828130 [ 3033 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:46.229973 [ 3021 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:46.279851 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:46.362892 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:46.477674 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:46.770240 [ 3046 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:46.779126 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:46.831048 [ 3047 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:46.856028 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:46.888986 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:47.232270 [ 3064 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:47.502645 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:47.670382 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:47.690353 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:47.855257 [ 3093 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:47.885534 [ 3088 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:48.233864 [ 3084 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:48.416789 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:48.534678 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:48.718484 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:48.858595 [ 3106 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:48.888522 [ 3103 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:49.079801 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:49.236571 [ 3110 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:49.271866 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:49.280893 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:49.791612 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:49.803107 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:49.860849 [ 3134 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:49.886785 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:49.891189 [ 3120 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:50.240339 [ 3151 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:50.815222 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:50.864258 [ 3149 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:50.920641 [ 3155 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:51.038828 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:51.063811 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:51.243449 [ 3143 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:51.431504 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:51.546657 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:51.592175 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:51.867035 [ 3183 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:51.924119 [ 3175 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:52.246561 [ 3185 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:52.399136 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:52.426069 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:52.441162 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:52.770179 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:52.786091 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:52.870142 [ 3201 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:52.927825 [ 3207 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:53.017773 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:53.249263 [ 3217 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:53.317075 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:53.640880 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:53.850025 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:53.874028 [ 3225 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:53.883451 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:53.931161 [ 3244 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:54.164701 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:54.176659 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:54.251231 [ 3246 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:54.400932 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:54.876995 [ 3255 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:54.939039 [ 3258 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:54.960464 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:55.096371 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:55.137642 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:55.256183 [ 2762 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:55.290872 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:55.718907 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:55.880013 [ 2774 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:55.929136 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:55.941091 [ 2753 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:55.966828 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:56.259947 [ 2765 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:56.882677 [ 2795 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:56.903611 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:56.940623 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:56.943387 [ 2803 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:56.959676 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:57.262827 [ 2805 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:57.359296 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:57.376366 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:57.376739 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:57.518770 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:57.885971 [ 2822 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:57.946256 [ 2833 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:58.178594 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:58.192843 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:58.265374 [ 2832 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:58.484305 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:58.691294 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:58.888197 [ 2856 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:58.948840 [ 2850 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:59.117323 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:59.134999 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:59.137738 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:59.268186 [ 2873 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:59.630921 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:59.643652 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:59.891088 [ 2871 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:29:59.951906 [ 2878 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:00.099510 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:00.120600 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:00.258429 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:00.270160 [ 2884 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:00.579249 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:00.780646 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:00.894401 [ 2898 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:00.955137 [ 2886 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:01.100309 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:01.254266 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:01.272641 [ 2913 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:01.279535 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:01.284940 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:01.882782 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:01.897440 [ 2920 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:01.958381 [ 2932 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:02.362734 [ 2940 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:02.551332 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:02.566878 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:02.842054 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:02.900342 [ 2958 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:02.961560 [ 2955 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:03.366333 [ 2956 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:03.507235 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:03.538306 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:03.734668 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:03.902911 [ 2969 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:04.045807 [ 2978 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:04.378200 [ 2986 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:04.591235 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:04.710529 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:04.738806 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:04.906012 [ 2993 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:05.044008 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:05.049580 [ 3005 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:05.063894 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:05.381657 [ 3015 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:05.520891 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:05.909410 [ 3022 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:06.053016 [ 3014 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:06.108896 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:06.126929 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:06.384597 [ 3027 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:06.527766 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:06.912383 [ 3029 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:06.953384 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:07.032406 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:07.054777 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:07.055849 [ 3047 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:07.387642 [ 3058 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:07.397876 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:07.549519 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:07.586494 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:07.918308 [ 3076 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:07.929667 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:07.979187 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:08.060184 [ 3072 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:08.233260 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:08.390547 [ 3089 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:08.792109 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:08.825659 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:08.922136 [ 3092 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:08.979501 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:09.063516 [ 3100 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:09.393732 [ 3077 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:09.811711 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:09.925526 [ 3116 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:10.006563 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:10.024546 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:10.066799 [ 3131 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:10.396948 [ 3134 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:10.694968 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:10.928882 [ 3136 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:11.021214 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:11.066331 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:11.070007 [ 3147 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:11.401161 [ 3142 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:11.525923 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:11.568441 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:11.585230 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:11.932691 [ 3177 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:12.072975 [ 3173 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:12.104274 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:12.126066 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:12.139868 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:12.404747 [ 3163 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:12.436795 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:12.960083 [ 3195 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:13.081608 [ 3201 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:13.151094 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:13.237694 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:13.271539 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:13.409285 [ 3209 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:13.966165 [ 3223 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:13.995070 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:14.083319 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:14.085285 [ 3215 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:14.091370 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:14.412694 [ 3244 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:14.492087 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:14.968755 [ 3243 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:15.102450 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:15.129954 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:15.142418 [ 3239 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:15.338489 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:15.416001 [ 3248 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:15.764013 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:15.822973 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:15.971848 [ 2760 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:16.145241 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:16.146262 [ 2754 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:16.419091 [ 2757 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:16.871363 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:16.955385 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:16.974999 [ 2789 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:17.004470 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:17.149135 [ 2788 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:17.421434 [ 2793 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:17.650079 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:17.684054 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:17.817669 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:17.967952 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:17.978296 [ 2796 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:17.989040 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:18.151765 [ 2805 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:18.424160 [ 2808 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:18.691018 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:18.706795 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:18.719981 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:18.981394 [ 2832 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:19.154978 [ 2835 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:19.430052 [ 2857 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:19.438884 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:19.518793 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:19.560460 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:19.984543 [ 2858 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:20.132292 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:20.145670 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:20.157749 [ 2855 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:20.432667 [ 2872 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:20.681751 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:20.987839 [ 2884 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:21.159898 [ 2896 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:21.356828 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:21.385196 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:21.415061 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:21.445450 [ 2895 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:21.990132 [ 2915 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:22.004389 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:22.162041 [ 2922 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:22.267656 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:22.448159 [ 2923 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:22.717671 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:22.744639 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:22.992599 [ 2935 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:23.164419 [ 2939 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:23.298561 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:23.452200 [ 2937 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:23.546512 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:23.644486 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:23.682266 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:23.892919 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:23.995057 [ 2966 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:24.231962 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:24.252035 [ 2971 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:24.431141 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:24.454705 [ 2972 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:24.458836 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:24.997470 [ 2983 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:25.224269 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:25.254239 [ 2993 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:25.463565 [ 2995 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:25.588996 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:25.598906 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:25.999786 [ 3017 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:26.041263 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:26.256387 [ 3016 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:26.412420 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:26.430197 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:26.556799 [ 3035 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:27.003014 [ 3037 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:27.179560 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:27.259165 [ 3046 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:27.365502 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:27.398403 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:27.559820 [ 3045 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:27.703986 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:27.949242 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:28.007009 [ 3076 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:28.060365 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:28.262352 [ 3074 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:28.510869 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:28.563000 [ 3088 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:28.966779 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:29.010068 [ 3095 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:29.183408 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:29.209801 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:29.265766 [ 3082 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:29.527249 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:29.566225 [ 3109 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:29.637800 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:29.667382 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:29.915539 [ 3353 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {02260661-95d0-4eca-a73d-2543d99ce5b9::202509_317_387_14}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:30.013212 [ 3114 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:30.229174 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:30.292746 [ 3129 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:30.573667 [ 3130 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:30.621263 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:30.643512 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:31.005627 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:31.016371 [ 3145 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:31.125826 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:31.174145 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:31.329010 [ 3158 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:31.540174 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:31.557852 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:31.579892 [ 3156 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:31.717691 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:31.938583 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:31.986348 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:32.007275 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:32.018440 [ 3167 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:32.332040 [ 3186 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:32.582792 [ 3195 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:32.933779 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:32.960199 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:32.976891 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:33.021188 [ 3205 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:33.334742 [ 3190 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:33.346625 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:33.393259 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:33.585952 [ 3226 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:33.804467 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:33.815151 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:33.851667 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:34.025087 [ 3238 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:34.337345 [ 3235 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:34.588918 [ 2745 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:34.637456 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:34.654861 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:34.900832 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:35.028304 [ 2749 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:35.209280 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:35.225252 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:35.339438 [ 2764 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:35.591713 [ 2768 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:35.886788 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:36.031049 [ 2781 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:36.259665 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:36.273656 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:36.341366 [ 2751 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:36.480154 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:36.594593 [ 2758 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:37.034166 [ 2798 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:37.277319 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:37.307821 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:37.344464 [ 2810 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:37.550622 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:37.598021 [ 2820 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:37.863913 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:37.885461 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:38.036319 [ 2809 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:38.347664 [ 2833 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:38.601426 [ 2848 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:38.606820 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:38.781405 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:39.039347 [ 2846 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:39.116695 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:39.153914 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:39.350509 [ 2863 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:39.466510 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:39.487087 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:39.604999 [ 2855 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:40.009860 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:40.042234 [ 2878 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:40.245083 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:40.251961 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:40.352970 [ 2889 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:40.539597 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:40.563838 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:40.608181 [ 2837 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:40.861987 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:40.896161 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:40.920033 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:41.045173 [ 2908 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:41.301696 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:41.316393 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:41.355847 [ 2922 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:41.611458 [ 2926 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:41.961902 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:42.057450 [ 2931 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:42.358157 [ 2942 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:42.445917 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:42.485430 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:42.622949 [ 2930 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:43.067475 [ 2949 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:43.078171 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:43.339723 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:43.361052 [ 2966 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:43.377453 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:43.626137 [ 2974 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:43.833379 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:44.071070 [ 2979 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:44.138613 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:44.368210 [ 2989 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:44.406730 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:44.504406 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:44.523088 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:44.628388 [ 2981 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:45.070757 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:45.073982 [ 2933 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:45.371106 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:45.371709 [ 3007 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:45.401069 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:45.630345 [ 3011 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:45.814128 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:45.844326 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:45.923490 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:46.078226 [ 3027 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:46.321259 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:46.384892 [ 3032 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:46.632961 [ 3040 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:47.039995 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:47.083509 [ 3059 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:47.090780 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:47.114160 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:47.389462 [ 3066 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:47.544896 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:47.574579 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:47.635963 [ 3068 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:47.716414 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:48.088015 [ 3081 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:48.246341 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:48.393051 [ 3092 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:48.638837 [ 3105 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:48.871691 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:48.982565 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:49.090003 [ 3113 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:49.203157 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:49.390892 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:49.395581 [ 3119 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:49.406616 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:49.425093 [ 3816 ] {38bb8910-8d69-4308-8cf4-e8824b6fa49f} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:52102) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:30:49.434765 [ 3816 ] {38bb8910-8d69-4308-8cf4-e8824b6fa49f} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:30:49.616650 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:49.641915 [ 3134 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:50.092709 [ 3136 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:50.095703 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:50.117193 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:50.188017 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:50.398095 [ 3147 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:50.461715 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:50.647307 [ 3158 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:50.914055 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:50.984759 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:51.096151 [ 3176 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:51.133167 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:51.401003 [ 3168 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:51.463541 [ 3816 ] {c0276b7f-06f6-4915-a5d7-30f8b7f87dfe} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:52118) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:30:51.468547 [ 3816 ] {c0276b7f-06f6-4915-a5d7-30f8b7f87dfe} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:30:51.598336 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:51.642746 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:51.649488 [ 3163 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:51.987025 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:51.999885 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:52.099280 [ 3194 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:52.164465 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:52.405302 [ 3169 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:52.642936 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:52.652358 [ 3216 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:52.667974 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:52.725045 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:52.901169 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:53.102675 [ 3220 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:53.408929 [ 3236 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:53.502425 [ 3816 ] {2aadb6c5-fd46-43c4-a582-b3c3d184e20f} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:52134) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:30:53.515746 [ 3816 ] {2aadb6c5-fd46-43c4-a582-b3c3d184e20f} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:30:53.655797 [ 3241 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:53.800600 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:53.871533 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:53.965892 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:54.105723 [ 3230 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:54.447572 [ 2747 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:54.659244 [ 2752 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:54.844928 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:54.866176 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:55.108217 [ 2757 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:55.152449 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:55.325965 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:55.450690 [ 2774 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:55.584206 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:55.662846 [ 2771 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:55.891075 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:55.906636 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:56.029363 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:56.114098 [ 2790 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:56.453484 [ 2804 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:56.461644 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:56.503866 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:56.525474 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:56.665987 [ 2805 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:56.975681 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:56.989265 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:57.116383 [ 2811 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:57.134460 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:57.456102 [ 2836 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:57.669183 [ 2824 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:57.962325 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:58.119009 [ 2850 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:58.236108 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:58.281779 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:58.459353 [ 2851 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:58.520529 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:58.672819 [ 2873 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:59.117857 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:59.121790 [ 2868 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:59.155643 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:59.461514 [ 2881 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:59.469770 [ 13110 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:59.485967 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:59.555621 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:59.570399 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:59.675910 [ 2889 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:59.923842 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:59.958249 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:30:59.992965 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:00.124875 [ 2902 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:00.215433 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:00.464283 [ 2916 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:00.679050 [ 2907 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:00.752483 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:00.802994 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:00.857691 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:01.121252 [ 3357 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {95e6584c-1281-4071-8ae7-ba1b29088860::202509_1_196_39}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:01.133160 [ 2909 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:01.466651 [ 2931 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:01.682391 [ 2939 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:01.828581 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:01.916275 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:01.995439 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:02.046863 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:02.135003 [ 2953 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:02.409101 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:02.469305 [ 2961 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:02.534153 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:02.685581 [ 2960 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:02.686951 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:03.137784 [ 2970 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:03.268705 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:03.472662 [ 2975 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:03.679930 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:03.687970 [ 2981 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:03.707058 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:04.085982 [ 3816 ] {8f5aee1e-133e-4b8e-a81c-b40381fdb0e6} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:53108) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:31:04.090331 [ 3816 ] {8f5aee1e-133e-4b8e-a81c-b40381fdb0e6} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:31:04.140951 [ 2933 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:04.208775 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:04.228628 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:04.308040 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:04.475781 [ 3012 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:04.491251 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:04.518883 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:04.690628 [ 3018 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:05.144967 [ 3035 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:05.325577 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:05.478911 [ 3031 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:05.693550 [ 3037 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:05.774805 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:05.925442 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:06.147926 [ 3050 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:06.169340 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:06.214756 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:06.484540 [ 3067 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:06.552819 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:06.702189 [ 3062 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:06.827595 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:06.896262 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:06.914407 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:07.002565 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:07.151120 [ 3085 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:07.181050 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:07.492580 [ 3100 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:07.704622 [ 3082 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:08.033352 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:08.058157 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:08.154525 [ 3108 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:08.301937 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:08.500070 [ 3131 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:08.573781 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:08.658718 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:08.707806 [ 3135 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:08.724494 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:09.164566 [ 3141 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:09.252400 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:09.502871 [ 3155 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:09.711078 [ 3157 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:09.714243 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:09.735838 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:10.172100 [ 3177 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:10.343076 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:10.510602 [ 3184 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:10.603553 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:10.623066 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:10.714351 [ 3196 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:10.750648 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:11.175738 [ 3195 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:11.513451 [ 3169 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:11.716552 [ 3192 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:11.789059 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:11.801453 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:11.890157 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:12.178626 [ 3212 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:12.510207 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:12.515663 [ 3222 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:12.539066 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:12.718813 [ 3241 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:12.946322 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:13.249727 [ 3242 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:13.420123 [ 3816 ] {be0b8613-782d-4c6d-a370-a11ad2691884} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=0): (while reading column current_database): (while reading from part /var/lib/clickhouse/store/f81/f81dbc1c-a1bd-4968-9514-f0c97137169a/202509_189_189_0/ in table system.query_log (f81dbc1c-a1bd-4968-9514-f0c97137169a) located on disk default of type local, from mark 0 with max_rows_to_read = 6): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55474) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. ./src/IO/ReadBuffer.h:0: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x000000002775f11b 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::CachedCompressedReadBuffer::nextImpl() @ 0x000000002c7f8e5d 14. DB::ReadBuffer::next() @ 0x000000000b528aeb 15. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x000000000b581216 16. ./contrib/llvm-project/libcxx/include/new:246: DB::SerializationLowCardinality::deserializeBinaryBulkStatePrefix(DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>*) const @ 0x0000000027b57db7 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::readPrefix(DB::NameAndTypePair const&, std::shared_ptr const&, std::shared_ptr&, std::function const&) @ 0x000000002c7b4491 18. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c7b3d56 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.cpp:0: DB::MergeTreeReaderCompactSingleBuffer::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7eb73e 20. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 21. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000002c7e0473 22. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbec3 23. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbd35 24. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 25. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:53: DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002dea5374 26. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 27. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 28. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 29. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 30. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 31. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 2025.09.25 21:31:13.431002 [ 3816 ] {be0b8613-782d-4c6d-a370-a11ad2691884} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=0): (while reading column current_database): (while reading from part /var/lib/clickhouse/store/f81/f81dbc1c-a1bd-4968-9514-f0c97137169a/202509_189_189_0/ in table system.query_log (f81dbc1c-a1bd-4968-9514-f0c97137169a) located on disk default of type local, from mark 0 with max_rows_to_read = 6): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. ./src/IO/ReadBuffer.h:0: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x000000002775f11b 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::CachedCompressedReadBuffer::nextImpl() @ 0x000000002c7f8e5d 14. DB::ReadBuffer::next() @ 0x000000000b528aeb 15. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x000000000b581216 16. ./contrib/llvm-project/libcxx/include/new:246: DB::SerializationLowCardinality::deserializeBinaryBulkStatePrefix(DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr&, std::unordered_map, std::hash, std::equal_to, std::allocator>>>*) const @ 0x0000000027b57db7 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::readPrefix(DB::NameAndTypePair const&, std::shared_ptr const&, std::shared_ptr&, std::function const&) @ 0x000000002c7b4491 18. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c7b3d56 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.cpp:0: DB::MergeTreeReaderCompactSingleBuffer::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7eb73e 20. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:118: DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x000000002c7c9d8a 21. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000002c7e0473 22. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbec3 23. ./build_docker/./src/Storages/MergeTree/MergeTreeRangeReader.cpp:0: DB::MergeTreeRangeReader::read(unsigned long, DB::MarkRanges&) @ 0x000000002c7dbd35 24. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:0: DB::MergeTreeReadTask::read(DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002c82d05b 25. ./src/Storages/MergeTree/MergeTreeSelectAlgorithms.h:53: DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&, DB::MergeTreeReadTask::BlockSizeParams const&) @ 0x000000002dea5374 26. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8242f9 27. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 28. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 29. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 30. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 31. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 2025.09.25 21:31:13.450063 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:13.493718 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:13.518528 [ 3252 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:13.721735 [ 2747 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:14.038584 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:14.218030 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:14.253501 [ 2754 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:14.255228 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:14.525643 [ 2772 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:14.725195 [ 2766 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:14.901866 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:15.323105 [ 2789 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:15.332606 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:15.355393 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:15.533802 [ 2777 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:15.671999 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:15.690488 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:15.727692 [ 2784 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:15.756491 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:16.326730 [ 2813 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:16.526412 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:16.536529 [ 2805 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:16.737475 [ 2808 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:16.836248 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:16.858347 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:16.898986 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:17.329650 [ 2848 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:17.516502 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:17.539983 [ 2853 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:17.740609 [ 2850 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:17.916699 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:18.038545 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:18.332694 [ 2847 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:18.492313 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:18.543403 [ 2869 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:18.744021 [ 2878 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:19.005681 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:19.232077 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:19.261974 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:19.335817 [ 2897 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:19.550802 [ 2886 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:19.600174 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:19.821503 [ 2899 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:19.828197 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:19.866673 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:20.276024 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:20.291626 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:20.337998 [ 2917 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:20.553944 [ 2924 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:20.700375 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:20.825157 [ 2931 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:21.341106 [ 2942 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:21.487660 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:21.533747 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:21.557134 [ 2943 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:21.701956 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:21.828640 [ 2955 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:22.102496 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:22.276665 [ 3816 ] {629ee79c-74c6-4963-846a-7e5fe30a5fb2} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:51342) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:31:22.289122 [ 3816 ] {629ee79c-74c6-4963-846a-7e5fe30a5fb2} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:31:22.344241 [ 2966 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:22.346421 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:22.472387 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:22.498235 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:22.559437 [ 2976 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:22.831633 [ 2970 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:23.006184 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:23.347273 [ 2977 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:23.561342 [ 2981 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:23.606556 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:23.634633 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:23.686734 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:23.833841 [ 2933 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:23.950174 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:23.962784 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:24.340051 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:24.350389 [ 3022 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:24.564035 [ 3014 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:24.602681 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:24.659957 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:24.836584 [ 3034 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:24.957540 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:24.974603 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:24.982634 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:25.353730 [ 3040 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:25.567453 [ 3043 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:25.740644 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:25.749066 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:25.843732 [ 3054 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:26.036355 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:26.356759 [ 3070 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:26.570130 [ 3068 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:26.660542 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:26.743565 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:26.778546 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:26.851169 [ 3085 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:27.360029 [ 3107 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:27.491554 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:27.572390 [ 3113 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:27.854763 [ 3108 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:27.955959 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:27.970594 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:28.136243 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:28.363244 [ 3129 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:28.575115 [ 3151 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:28.773258 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:28.864864 [ 3146 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:28.961088 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:29.070318 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:29.369310 [ 3144 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:29.578030 [ 3161 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:29.638517 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:29.867432 [ 3171 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:30.214333 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:30.362194 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:30.372044 [ 3185 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:30.376444 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:30.580884 [ 3189 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:30.787623 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:30.817191 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:30.870563 [ 3194 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:31.061118 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:31.375194 [ 3198 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:31.587858 [ 3214 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:31.617473 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:31.731726 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:31.741389 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:31.872900 [ 3208 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:32.076376 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:32.104897 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:32.378030 [ 3225 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:32.591078 [ 3229 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:32.636440 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:32.710150 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:32.746983 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:32.875904 [ 3249 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:33.330731 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:33.354548 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:33.380944 [ 2746 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:33.524870 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:33.525607 [ 3357 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {261a6bdf-2436-455f-b6df-dbd9b591b8be::202509_1_211_94}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001adfc17a 4. ./build_docker/./src/Common/MemoryTracker.cpp:187: MemoryTracker::injectFault() const @ 0x000000001adf8d1a 5. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x000000002cc30a65 6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f47b 7. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:33.594139 [ 2759 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:33.852885 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:33.880201 [ 3259 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:33.889754 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:34.369998 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:34.383623 [ 2774 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:34.461266 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:34.486671 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:34.597275 [ 2771 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:34.924743 [ 2751 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:35.126181 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:35.387739 [ 2782 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:35.599752 [ 2797 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:35.642419 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:35.665019 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:35.934485 [ 2813 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:35.994817 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:36.287437 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:36.355018 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:36.396454 [ 2830 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:36.582077 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:36.606597 [ 2819 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:36.694036 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:36.748548 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:36.863162 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:36.937020 [ 2845 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:37.322501 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:37.399803 [ 2862 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:37.440956 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:37.610083 [ 2847 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:37.941710 [ 2866 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:38.032240 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:38.402949 [ 2890 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:38.445588 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:38.613600 [ 2877 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:38.628632 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:38.653981 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:38.953230 [ 2891 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:39.255162 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:39.273754 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:39.370353 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:39.405181 [ 2913 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:39.645652 [ 2921 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:39.956606 [ 2923 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:40.020254 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:40.038046 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:40.324712 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:40.400319 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:40.408543 [ 2942 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:40.416765 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:40.648803 [ 2943 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:40.852123 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:40.960129 [ 2953 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:41.415527 [ 2956 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:41.607613 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:41.651783 [ 2967 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:41.652290 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:41.798917 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:41.963700 [ 2985 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:42.111161 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:42.418517 [ 2982 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:42.505740 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:42.534231 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:42.607500 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:42.655003 [ 2998 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:42.825954 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:42.854986 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:42.966294 [ 3007 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:43.022740 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:43.189585 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:43.208026 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:43.278659 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:43.421752 [ 3025 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:43.666050 [ 3816 ] {8bc4efaa-fcfc-49ab-9aba-09445b4836e8} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:33182) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:31:43.670882 [ 3816 ] {8bc4efaa-fcfc-49ab-9aba-09445b4836e8} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:31:43.658042 [ 3021 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:43.969333 [ 3031 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:44.075322 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:44.085897 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:44.117058 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:44.424933 [ 3039 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:44.751606 [ 3053 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:44.891873 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:44.972570 [ 3056 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:44.996552 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:45.079870 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:45.317136 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:45.428662 [ 3062 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:45.639830 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:45.666885 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:45.755006 [ 3068 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:45.974693 [ 3091 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:46.025403 [ 3816 ] {f3cb4e69-5fae-41d9-8334-638498321857} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=0): (while reading column current_database): (while reading from part /var/lib/clickhouse/store/f81/f81dbc1c-a1bd-4968-9514-f0c97137169a/202509_1_188_65/ in table system.query_log (f81dbc1c-a1bd-4968-9514-f0c97137169a) located on disk default of type local, from mark 0 with max_rows_to_read = 8071): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:33196) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. ./src/IO/ReadBuffer.h:0: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x000000002775f11b 13. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:22: DB::CompressedReadBufferFromFile::nextImpl() @ 0x0000000027763d41 14. DB::ReadBuffer::next() @ 0x000000000b528aeb 15. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:168: DB::MergeTreeMarksLoader::loadMarksImpl() @ 0x000000002c763f74 16. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::MergeTreeMarksLoader::loadMarksSync() @ 0x000000002c761dcf 17. ./src/Common/threadPoolCallbackRunner.h:53: std::__packaged_task_func> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)::operator()(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)::'lambda'(), std::allocator> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)::operator()(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)::'lambda'()>, std::shared_ptr ()>::operator()() @ 0x000000002c76695b 18. ./contrib/llvm-project/libcxx/include/future:1953: std::packaged_task ()>::operator()() @ 0x000000002c76bbb3 19. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 20. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:31:46.051788 [ 3816 ] {f3cb4e69-5fae-41d9-8334-638498321857} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=0): (while reading column current_database): (while reading from part /var/lib/clickhouse/store/f81/f81dbc1c-a1bd-4968-9514-f0c97137169a/202509_1_188_65/ in table system.query_log (f81dbc1c-a1bd-4968-9514-f0c97137169a) located on disk default of type local, from mark 0 with max_rows_to_read = 8071): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. ./src/IO/ReadBuffer.h:0: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x000000002775f11b 13. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:22: DB::CompressedReadBufferFromFile::nextImpl() @ 0x0000000027763d41 14. DB::ReadBuffer::next() @ 0x000000000b528aeb 15. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:168: DB::MergeTreeMarksLoader::loadMarksImpl() @ 0x000000002c763f74 16. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::MergeTreeMarksLoader::loadMarksSync() @ 0x000000002c761dcf 17. ./src/Common/threadPoolCallbackRunner.h:53: std::__packaged_task_func> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)::operator()(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)::'lambda'(), std::allocator> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)::operator()(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)::'lambda'()>, std::shared_ptr ()>::operator()() @ 0x000000002c76695b 18. ./contrib/llvm-project/libcxx/include/future:1953: std::packaged_task ()>::operator()() @ 0x000000002c76bbb3 19. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 20. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:31:46.184665 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:46.218236 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:46.341037 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:46.431885 [ 3086 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:46.637683 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:46.758495 [ 3082 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:46.928782 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:46.975725 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:46.977245 [ 3104 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:47.434633 [ 3121 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:47.609833 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:47.761854 [ 3128 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:47.980494 [ 3136 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:48.033628 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:48.053427 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:48.437712 [ 3140 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:48.635839 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:48.765292 [ 3161 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:48.983742 [ 3177 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:49.191128 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:49.222833 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:49.440561 [ 3181 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:49.664562 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:49.768950 [ 3167 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:49.987048 [ 3195 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:50.032640 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:50.220902 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:50.249863 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:50.443614 [ 3210 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:50.771725 [ 3205 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:50.989367 [ 3208 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:51.107022 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:51.211722 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:51.242691 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:51.446825 [ 3231 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:51.774370 [ 3240 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:51.882503 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:52.002377 [ 3235 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:52.239300 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:52.282334 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:52.361165 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:52.449527 [ 3245 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:52.781756 [ 3257 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:52.953506 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:53.005982 [ 2756 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:53.220419 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:53.330525 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:53.350661 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:53.368319 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:53.452585 [ 2763 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:53.673248 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:53.710088 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:53.786221 [ 2789 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:54.008647 [ 2783 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:54.093747 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:54.108760 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:54.393247 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:54.455938 [ 2790 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:54.555872 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:54.788545 [ 2799 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:54.966718 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:54.993393 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:55.011826 [ 2804 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:55.068640 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:55.458705 [ 2808 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:55.543852 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:55.559252 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:55.790846 [ 2816 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:55.882285 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:56.014547 [ 2836 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:56.440855 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:56.460733 [ 2823 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:56.478472 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:56.793312 [ 2852 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:56.861647 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:56.986016 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:57.008091 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:57.016640 [ 2855 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:57.356792 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:57.466200 [ 2871 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:57.795705 [ 2892 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:57.928928 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:57.938574 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:58.018810 [ 2901 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:58.125619 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:58.382494 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:58.468491 [ 2904 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:58.798429 [ 2910 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:59.019891 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:59.021596 [ 2921 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:59.224563 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:59.247739 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:59.441783 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:59.471647 [ 2925 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:59.800888 [ 2942 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:59.808912 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:31:59.829244 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:00.028972 [ 2930 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:00.158061 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:00.474963 [ 2951 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:00.605205 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:00.624564 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:00.628263 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:00.804026 [ 2964 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:01.030192 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:01.078512 [ 2974 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:01.456127 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:01.535195 [ 2972 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:01.623856 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:01.784433 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:01.806346 [ 2991 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:01.840708 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:02.081420 [ 2981 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:02.537593 [ 3015 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:02.568379 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:02.810141 [ 3022 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:02.846298 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:02.862333 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:03.085791 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:03.103760 [ 3026 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:03.412767 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:03.540365 [ 3032 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:03.812460 [ 3037 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:03.974358 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:03.991393 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:04.105843 [ 3049 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:04.381855 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:04.542787 [ 3064 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:04.817388 [ 3060 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:05.070930 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:05.108499 [ 3070 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:05.141675 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:05.165946 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:05.544966 [ 3071 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:05.603894 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:05.819492 [ 3099 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:06.111696 [ 3102 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:06.198057 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:06.224278 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:06.251780 [ 3816 ] {bbcb50ad-2ee6-40f6-a39d-a5572aa278de} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:54998) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:32:06.262951 [ 3816 ] {bbcb50ad-2ee6-40f6-a39d-a5572aa278de} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:32:06.419574 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:06.547722 [ 3113 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:06.638948 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:06.758741 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:06.819279 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:06.822599 [ 3110 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:07.114085 [ 3120 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:07.306395 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:07.338826 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:07.353111 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:07.550146 [ 3139 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:07.826078 [ 3149 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:07.903337 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:07.920598 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:08.124424 [ 3144 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:08.359929 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:08.552243 [ 3171 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:08.677591 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:08.784249 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:08.840082 [ 3170 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:09.127223 [ 3175 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:09.308817 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:09.321810 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:09.384119 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:09.554712 [ 3195 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:09.842324 [ 3169 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:09.860563 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:09.875205 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:10.135292 [ 3193 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:10.294708 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:10.339342 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:10.344348 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:10.556872 [ 3215 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:10.794369 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:10.846713 [ 3231 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:11.138501 [ 3232 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:11.267351 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:11.290176 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:11.295107 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:11.559657 [ 3230 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:11.564022 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:11.584929 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:11.605654 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:11.849042 [ 3260 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:12.080526 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:12.116098 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:12.140925 [ 2756 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:12.170022 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:12.515812 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:12.562625 [ 2766 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:12.876259 [ 2755 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:13.102767 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:13.129244 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:13.143348 [ 2777 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:13.323712 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:13.565360 [ 2793 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:13.878903 [ 2791 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:13.971701 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:13.997981 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:14.016544 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:14.031380 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:14.147543 [ 2815 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:14.568380 [ 2809 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:14.825355 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:14.882314 [ 2833 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:15.150759 [ 2834 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:15.237019 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:15.253365 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:15.484999 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:15.571473 [ 2823 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:15.841091 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:15.884575 [ 2852 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:16.153981 [ 2862 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:16.273411 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:16.294145 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:16.378897 [ 3816 ] {a5da6aa4-e3d7-4f0d-8828-ce2f83863970} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:58488) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:32:16.383356 [ 3816 ] {a5da6aa4-e3d7-4f0d-8828-ce2f83863970} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:32:16.542518 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:16.574806 [ 2878 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:16.646895 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:16.862088 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:16.887195 [ 2879 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:16.961584 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:17.198318 [ 2896 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:17.219151 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:17.577241 [ 2898 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:17.693409 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:17.788907 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:17.805760 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:17.890369 [ 2903 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:18.172665 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:18.200909 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:18.201052 [ 2926 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:18.580337 [ 2928 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:18.628468 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:18.663391 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:18.799853 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:18.896375 [ 2942 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:19.046001 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:19.068165 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:19.205302 [ 2947 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:19.371628 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:19.583592 [ 2958 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:19.866935 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:19.902421 [ 2956 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:19.925655 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:19.998631 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:20.208363 [ 2974 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:20.381660 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:20.418528 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:20.586899 [ 2985 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:20.776655 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:20.907205 [ 2989 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:21.055209 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:21.212706 [ 2991 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:21.589626 [ 3000 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:21.593175 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:21.623394 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:21.828415 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:21.909497 [ 3013 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:22.179181 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:22.215122 [ 3025 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:22.244640 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:22.271172 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:22.323182 [ 3816 ] {983ac9f7-3ab6-48bc-a39c-3879975fabdf} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:47946) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:32:22.342344 [ 3816 ] {983ac9f7-3ab6-48bc-a39c-3879975fabdf} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:32:22.591781 [ 3030 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:22.911611 [ 3036 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:23.118224 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:23.126032 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:23.140298 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:23.218193 [ 3039 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:23.594010 [ 3053 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:23.701533 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:23.768073 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:23.778221 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:23.913654 [ 3063 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:24.109114 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:24.132058 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:24.220433 [ 3065 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:24.500580 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:24.596802 [ 3083 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:24.642137 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:24.654098 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:24.915897 [ 3085 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:25.222751 [ 3092 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:25.266272 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:25.362185 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:25.423587 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:25.669995 [ 3107 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:25.693105 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:25.873204 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:25.896180 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:25.919851 [ 3108 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:26.201101 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:26.224650 [ 3115 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:26.487826 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:26.499320 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:26.672013 [ 3135 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:26.922852 [ 3139 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:27.227862 [ 3147 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:27.290705 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:27.678339 [ 3144 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:27.782986 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:27.819615 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:27.925696 [ 3165 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:28.202388 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:28.231070 [ 3170 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:28.233459 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:28.424573 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:28.681445 [ 3097 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:28.928104 [ 3187 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:29.234695 [ 3199 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:29.615440 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:29.683707 [ 3210 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:29.705323 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:29.730511 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:29.929843 [ 3204 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:30.174976 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:30.237314 [ 3219 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:30.686581 [ 3231 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:30.792206 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:30.823055 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:30.932689 [ 3237 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:31.056896 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:31.210389 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:31.238058 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:31.240324 [ 3247 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:31.694594 [ 2745 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:31.816304 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:31.935033 [ 2759 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:32.277166 [ 2767 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:32.342144 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:32.373623 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:32.394364 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:32.680293 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:32.723720 [ 2753 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:32.862188 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:32.884502 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:32.936931 [ 2789 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:33.280177 [ 2765 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:33.383391 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:33.686001 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:33.732028 [ 2799 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:33.878144 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:33.939981 [ 2803 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:33.976522 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:34.283340 [ 2805 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:34.363272 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:34.404602 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:34.682884 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:34.735527 [ 2811 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:34.920571 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:34.945263 [ 2834 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:34.969743 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:35.046879 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:35.286409 [ 2824 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:35.738807 [ 2850 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:35.815525 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:35.948534 [ 2842 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:36.008627 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:36.236075 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:36.289433 [ 2859 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:36.741142 [ 2878 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:36.741621 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:36.918450 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:36.951598 [ 2884 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:37.047718 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:37.069031 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:37.103725 [ 3816 ] {25dbbd87-ca6c-426d-a9c0-94bd98913bc8} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:41908) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:32:37.108032 [ 3816 ] {25dbbd87-ca6c-426d-a9c0-94bd98913bc8} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:32:37.292652 [ 2837 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:37.548666 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:37.743096 [ 2906 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:37.896895 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:37.915071 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:37.953786 [ 2903 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:38.108438 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:38.295016 [ 2914 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:38.745345 [ 2909 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:38.955536 [ 2931 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:38.969474 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:38.988461 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:39.006580 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:39.296763 [ 2826 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:39.616309 [ 13110 ] {} DynamicQueryHandler: Code: 452. DB::Exception: Setting max_execution_time shouldn't be greater than 60. (SETTING_CONSTRAINT_VIOLATION), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception>&, String>(int, FormatStringHelperImpl>&>::type, std::type_identity::type>, std::basic_string_view>&, String&&) @ 0x0000000026f841fa 4. ./build_docker/./src/Access/SettingsConstraints.cpp:368: DB::SettingsConstraints::Checker::check(DB::SettingChange&, DB::Field const&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f81de0 5. ./build_docker/./src/Access/SettingsConstraints.cpp:288: DB::SettingsConstraints::checkImpl(DB::Settings const&, DB::SettingChange&, DB::SettingsConstraints::ReactionOnViolation, DB::SettingSource) const @ 0x0000000026f802d1 6. ./contrib/llvm-project/libcxx/include/__algorithm/find_if.h:25: DB::SettingsConstraints::check(DB::Settings const&, DB::SettingsChanges&, DB::SettingSource) const @ 0x0000000026f808a4 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::Context::checkSettingsConstraintsWithLock(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853ad81 8. ./src/Common/SharedMutexHelper.h:106: DB::Context::checkSettingsConstraints(DB::SettingsChanges&, DB::SettingSource) @ 0x000000002853b971 9. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d065739 10. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 11. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 12. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 14. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 15. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 16. asan_thread_start(void*) @ 0x000000000aa43059 17. ? @ 0x00007f12cd75eac3 18. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:39.660622 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:39.681562 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:39.747643 [ 2937 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:39.920953 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:39.958687 [ 2953 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:40.164272 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:40.233287 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:40.299700 [ 2973 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:40.351063 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:40.573458 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:40.579274 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:40.621973 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:40.750364 [ 2986 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:40.961753 [ 2977 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:41.303137 [ 2991 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:41.426247 [ 3816 ] {c398655c-c29d-4e1c-89e3-65ad98dc3681} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:41924) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:32:41.429971 [ 3816 ] {c398655c-c29d-4e1c-89e3-65ad98dc3681} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:32:41.550486 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:41.753878 [ 3007 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:41.800509 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:41.818633 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:41.964882 [ 3012 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:42.104750 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:42.306686 [ 3026 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:42.756322 [ 3031 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:42.829116 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:42.831843 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:42.852690 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:42.968005 [ 3028 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:43.134207 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:43.309556 [ 3046 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:43.758523 [ 3056 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:43.912993 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:43.971148 [ 3054 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:44.060609 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:44.166520 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:44.312545 [ 3080 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:44.760028 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:44.761587 [ 3068 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:44.966744 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:44.973487 [ 3093 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:44.979623 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:45.077891 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:45.315985 [ 3092 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:45.510202 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:45.766487 [ 3100 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:45.976098 [ 3104 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:46.064725 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:46.089754 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:46.319484 [ 3116 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:46.629567 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:46.774985 [ 3126 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:46.857502 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:46.978173 [ 3120 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:47.322571 [ 3137 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:47.404763 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:47.425476 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:47.777820 [ 3152 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:47.971137 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:47.980898 [ 3153 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:48.073558 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:48.100077 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:48.325311 [ 3168 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:48.781082 [ 3184 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:48.870565 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:48.984027 [ 3189 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:49.166475 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:49.217839 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:49.332629 [ 3191 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:49.784109 [ 3203 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:49.788703 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:49.987139 [ 3210 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:50.307097 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:50.326594 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:50.335326 [ 3219 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:50.787136 [ 3244 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:50.790844 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:51.062299 [ 3241 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:51.233649 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:51.262141 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:51.337437 [ 3233 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:51.790755 [ 2747 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:51.868720 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:52.064440 [ 2749 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:52.302930 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:52.329180 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:52.436223 [ 2764 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:52.794222 [ 2773 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:52.849375 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:53.039934 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:53.066286 [ 2775 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:53.111152 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:53.131452 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:53.439006 [ 2758 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:53.600353 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:53.622639 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:53.796394 [ 2784 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:54.069036 [ 2798 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:54.126608 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:54.442471 [ 2804 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:54.799144 [ 2808 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:54.891349 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:54.896472 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:54.926899 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:55.071254 [ 2818 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:55.445770 [ 2819 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:55.617397 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:55.629099 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:55.803381 [ 2835 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:55.832587 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:56.073874 [ 2850 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:56.290971 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:56.313610 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:56.451106 [ 2873 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:56.660627 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:56.671974 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:56.808234 [ 2870 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:56.862834 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:57.077301 [ 2869 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:57.305317 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:57.320221 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:57.454485 [ 2889 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:57.813349 [ 2886 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:57.817130 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:57.978295 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:57.988277 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:58.080105 [ 2916 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:58.456601 [ 2913 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:58.723876 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:58.812760 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:58.816777 [ 2920 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:58.982318 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:59.007885 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:59.082547 [ 2931 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:59.431103 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:59.459160 [ 2941 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:59.738559 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:32:59.819809 [ 2950 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:00.085977 [ 2958 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:00.232435 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:00.313951 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:00.462705 [ 2964 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:00.802148 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:00.821841 [ 2967 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:01.096849 [ 2985 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:01.465727 [ 2992 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:01.495131 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:01.517090 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:01.750529 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:01.824541 [ 2996 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:01.954053 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:01.968502 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:02.100607 [ 3008 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:02.253213 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:02.471234 [ 3010 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:02.550550 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:02.723732 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:02.826906 [ 3014 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:03.108910 [ 3027 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:03.300484 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:03.352778 [ 3816 ] {fdff4cc7-832b-4c57-b4fe-b96534572002} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=2): (while reading column current_database): (while reading from part /var/lib/clickhouse/store/f81/f81dbc1c-a1bd-4968-9514-f0c97137169a/202509_1_198_75/ in table system.query_log (f81dbc1c-a1bd-4968-9514-f0c97137169a) located on disk default of type local, from mark 0 with max_rows_to_read = 8071): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:60872) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. ./src/IO/ReadBuffer.h:0: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x000000002775f11b 13. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:22: DB::CompressedReadBufferFromFile::nextImpl() @ 0x0000000027763d41 14. DB::ReadBuffer::next() @ 0x000000000b528aeb 15. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:168: DB::MergeTreeMarksLoader::loadMarksImpl() @ 0x000000002c763f74 16. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::MergeTreeMarksLoader::loadMarksSync() @ 0x000000002c761dcf 17. ./src/Common/threadPoolCallbackRunner.h:53: std::__packaged_task_func> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)::operator()(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)::'lambda'(), std::allocator> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)::operator()(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)::'lambda'()>, std::shared_ptr ()>::operator()() @ 0x000000002c76695b 18. ./contrib/llvm-project/libcxx/include/future:1953: std::packaged_task ()>::operator()() @ 0x000000002c76bbb3 19. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 20. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:33:03.358922 [ 3816 ] {fdff4cc7-832b-4c57-b4fe-b96534572002} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=6, jobs=2): (while reading column current_database): (while reading from part /var/lib/clickhouse/store/f81/f81dbc1c-a1bd-4968-9514-f0c97137169a/202509_1_198_75/ in table system.query_log (f81dbc1c-a1bd-4968-9514-f0c97137169a) located on disk default of type local, from mark 0 with max_rows_to_read = 8071): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000266005a2 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x00000000265fffb4 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x00000000265fd4e7 9. ./build_docker/./src/IO/AsynchronousReadBufferFromFileDescriptor.cpp:62: DB::AsynchronousReadBufferFromFileDescriptor::asyncReadInto(char*, unsigned long, Priority) @ 0x00000000265e6fbd 10. ./contrib/llvm-project/libcxx/include/future:1123: DB::AsynchronousReadBufferFromFileDescriptor::nextImpl() @ 0x00000000265e7bb3 11. DB::ReadBuffer::next() @ 0x000000000b528aeb 12. ./src/IO/ReadBuffer.h:0: DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) @ 0x000000002775f11b 13. ./build_docker/./src/Compression/CompressedReadBufferFromFile.cpp:22: DB::CompressedReadBufferFromFile::nextImpl() @ 0x0000000027763d41 14. DB::ReadBuffer::next() @ 0x000000000b528aeb 15. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:168: DB::MergeTreeMarksLoader::loadMarksImpl() @ 0x000000002c763f74 16. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::MergeTreeMarksLoader::loadMarksSync() @ 0x000000002c761dcf 17. ./src/Common/threadPoolCallbackRunner.h:53: std::__packaged_task_func> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)::operator()(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)::'lambda'(), std::allocator> (DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)> DB::threadPoolCallbackRunnerUnsafe, DB::MergeTreeMarksLoader::loadMarksAsync()::$_0>(ThreadPoolImpl>&, String const&)::'lambda'(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)::operator()(DB::MergeTreeMarksLoader::loadMarksAsync()::$_0&&, Priority)::'lambda'()>, std::shared_ptr ()>::operator()() @ 0x000000002c76695b 18. ./contrib/llvm-project/libcxx/include/future:1953: std::packaged_task ()>::operator()() @ 0x000000002c76bbb3 19. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 20. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 23. asan_thread_start(void*) @ 0x000000000aa43059 24. ? @ 0x00007f12cd75eac3 25. ? @ 0x00007f12cd7f0850 2025.09.25 21:33:03.475752 [ 3028 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:03.533529 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:03.695500 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:03.794889 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:03.812499 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:03.832642 [ 3052 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:03.999428 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:04.112965 [ 3060 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:04.204253 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:04.229049 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:04.488787 [ 3076 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:04.834921 [ 3073 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:05.116091 [ 3093 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:05.129468 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:05.390286 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:05.454895 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:05.494507 [ 3090 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:05.838285 [ 3082 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:05.952230 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:06.119095 [ 3077 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:06.190479 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:06.208758 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:06.504432 [ 3110 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:06.559111 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:06.727014 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:06.752437 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:06.841608 [ 3138 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:06.907537 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:07.122386 [ 3150 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:07.527214 [ 3154 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:07.845083 [ 3153 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:07.906911 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:07.909821 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:07.925590 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:08.125388 [ 3170 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:08.530368 [ 3184 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:08.698253 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:08.848329 [ 3186 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:08.962470 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:08.976544 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:09.128572 [ 3198 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:09.153617 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:09.533395 [ 3214 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:09.851356 [ 3216 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:09.907555 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:10.006339 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:10.047269 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:10.131571 [ 3219 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:10.341470 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:10.439550 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:10.545145 [ 3244 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:10.739205 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:10.854823 [ 3238 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:10.935558 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:10.952096 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:11.134500 [ 3242 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:11.547617 [ 3255 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:11.593623 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:11.858736 [ 3234 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:12.127782 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:12.137448 [ 2754 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:12.154067 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:12.359801 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:12.450642 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:12.461667 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:12.555730 [ 2774 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:12.861777 [ 2771 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:13.140411 [ 2765 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:13.295657 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:13.307204 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:13.349856 [ 3816 ] {10fa3afa-a52b-4647-9cc3-c5bbe02bef9a} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=735). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50474) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:33:13.359191 [ 3816 ] {10fa3afa-a52b-4647-9cc3-c5bbe02bef9a} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=735). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:33:13.418943 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:13.559093 [ 2797 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:13.711283 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:13.777685 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:13.785728 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:13.864718 [ 2815 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:13.878995 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:14.143272 [ 2806 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:14.188391 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:14.206831 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:14.559229 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:14.562641 [ 2833 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:14.663881 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:14.683043 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:14.866944 [ 2838 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:15.146462 [ 2843 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:15.179875 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:15.212580 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:15.566550 [ 2853 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:15.665450 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:15.748717 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:15.770562 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:15.869111 [ 2864 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:16.148838 [ 2865 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:16.502690 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:16.533916 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:16.569881 [ 2881 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:16.576446 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:16.872086 [ 2877 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:17.149413 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:17.152886 [ 2902 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:17.262013 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:17.379390 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:17.573857 [ 2905 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:17.967138 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:17.971462 [ 2910 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:18.155511 [ 2926 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:18.429213 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:18.543999 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:18.579821 [ 2931 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:18.866560 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:18.974621 [ 2826 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:19.117502 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:19.142530 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:19.158134 [ 2938 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:19.539775 [ 3816 ] {59834bd9-2f20-4567-bfe7-87e47dfe27d1} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=735). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:33022) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:33:19.552031 [ 3816 ] {59834bd9-2f20-4567-bfe7-87e47dfe27d1} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=772, jobs=735). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d36920d 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:138: DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x000000002d369e5f 8. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18af7b 9. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 10. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 11. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 13. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 14. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 15. asan_thread_start(void*) @ 0x000000000aa43059 16. ? @ 0x00007f12cd75eac3 17. ? @ 0x00007f12cd7f0850 2025.09.25 21:33:19.582286 [ 2955 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:19.889302 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:19.977876 [ 2956 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:20.146894 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:20.160707 [ 2960 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:20.187414 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:20.584895 [ 2979 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:20.770707 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:20.981027 [ 2989 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:21.043477 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:21.063202 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:21.163551 [ 2982 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:21.220194 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:21.392342 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:21.588428 [ 3015 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:21.977232 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:21.983824 [ 3020 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:22.166882 [ 3011 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:22.168851 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:22.221776 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:22.591910 [ 3034 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:22.825982 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:22.991784 [ 3037 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:23.169841 [ 3044 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:23.286154 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:23.382396 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:23.405616 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:23.595028 [ 3058 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:23.994842 [ 3060 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:24.018490 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:24.031246 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:24.190876 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:24.195581 [ 3069 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:24.598531 [ 3073 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:24.998017 [ 3084 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:25.074316 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:25.198879 [ 3086 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:25.216814 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:25.261367 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:25.329118 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:25.602192 [ 3106 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:25.845113 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:25.880240 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:26.038194 [ 3117 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:26.201946 [ 3121 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:26.406976 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:26.605933 [ 3129 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:26.812815 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:26.932674 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:27.040326 [ 3136 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:27.204895 [ 3147 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:27.322252 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:27.392318 [ 3816 ] {37d77ee1-f492-43b4-9d66-fa181fd8aea0} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43532) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:33:27.399999 [ 3816 ] {37d77ee1-f492-43b4-9d66-fa181fd8aea0} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=2): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:33:27.609264 [ 3164 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:27.760284 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:27.770894 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:28.011886 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:28.043148 [ 3170 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:28.147521 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:28.163291 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:28.234608 [ 3181 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:28.613988 [ 3188 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:28.685727 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:28.901905 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:29.046687 [ 3179 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:29.060453 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:29.071815 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:29.237161 [ 3193 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:29.465064 [ 3816 ] {dbccc67c-e899-4adf-81f7-8fc026ebf6e8} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43542) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:33:29.468347 [ 3816 ] {dbccc67c-e899-4adf-81f7-8fc026ebf6e8} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:33:29.616301 [ 3206 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:29.775068 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:29.794973 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:29.935903 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:30.048999 [ 3244 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:30.242886 [ 3240 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:30.618994 [ 3229 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:30.945563 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:31.000457 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:31.031666 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:31.051748 [ 3239 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:31.081501 [ 3816 ] {8e495a29-a811-4765-8504-9904c93afec5} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43546) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:33:31.086599 [ 3816 ] {8e495a29-a811-4765-8504-9904c93afec5} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:33:31.245553 [ 3245 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:31.622575 [ 2749 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:31.668278 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:31.695054 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:32.034781 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:32.054901 [ 2768 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:32.186481 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:32.205204 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:32.290669 [ 2757 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:32.625328 [ 2770 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:33.057913 [ 2785 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:33.090398 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:33.293973 [ 2782 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:33.356693 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:33.390023 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:33.630426 [ 2799 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:33.917784 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:33.928256 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:34.060979 [ 2815 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:34.275474 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:34.327377 [ 2808 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:34.603244 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:34.633716 [ 2816 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:34.645333 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:35.022942 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:35.061181 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:35.066954 [ 2832 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:35.107649 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:35.413791 [ 2844 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:35.417595 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:35.636995 [ 2852 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:35.670229 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:35.894192 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:36.070419 [ 2847 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:36.317514 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:36.341865 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:36.353571 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:36.417307 [ 2876 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:36.552847 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:36.640143 [ 2881 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:37.118699 [ 3816 ] {7d747bdb-e8e3-4451-981a-c79525ad078b} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50482) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:33:37.124581 [ 3816 ] {7d747bdb-e8e3-4451-981a-c79525ad078b} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::addStream(DB::ISerialization::SubstreamPath const&, String const&) @ 0x000000002c802c31 10. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: void std::__function::__policy_invoker::__call_impl const&)::$_0, void (DB::ISerialization::SubstreamPath const&)>>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000002c804a95 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x0000000027a9e455 12. ./build_docker/./src/DataTypes/Serializations/ISerialization.cpp:106: DB::ISerialization::enumerateStreams(std::function const&, std::shared_ptr const&, COW::immutable_ptr const&) const @ 0x0000000027a9ea65 13. ./contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:100: DB::MergeTreeReaderWide::addStreams(DB::NameAndTypePair const&, std::shared_ptr const&) @ 0x000000002c7fe815 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderWide.cpp:0: DB::MergeTreeReaderWide::MergeTreeReaderWide(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7fdeea 15. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) @ 0x000000002c5eb9c6 16. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWide.cpp:43: DB::MergeTreeDataPartWide::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5e354e 17. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 18. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 19. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 20. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 21. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 22. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 23. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 24. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 25. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 26. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 27. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 28. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 29. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 30. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 31. asan_thread_start(void*) @ 0x000000000aa43059 2025.09.25 21:33:37.078035 [ 2882 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:37.366165 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:37.420193 [ 2902 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:37.506994 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:37.536904 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:37.642512 [ 2903 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:38.059432 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:38.073021 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:38.158317 [ 2924 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:38.404791 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:38.422995 [ 2914 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:38.645396 [ 2935 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:38.720156 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:38.948287 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:38.963379 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:39.202817 [ 2950 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:39.272945 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:39.426181 [ 2955 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:39.648372 [ 2959 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:39.719705 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:40.130339 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:40.165029 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:40.206288 [ 2987 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:40.428976 [ 2978 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:40.651994 [ 2980 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:40.757862 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:40.764932 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:40.770363 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:41.019774 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:41.094947 [ 3816 ] {1b9230ca-6809-4d0c-a915-eeb9f377991b} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:50492) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:33:41.099690 [ 3816 ] {1b9230ca-6809-4d0c-a915-eeb9f377991b} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:33:41.212940 [ 2996 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:41.252125 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:41.432181 [ 3000 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:41.487153 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:41.509797 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:41.655477 [ 3007 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:41.993121 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:42.219687 [ 3014 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:42.299698 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:42.344303 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:42.435555 [ 3034 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:42.658831 [ 3037 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:42.696707 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:42.978998 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:43.003022 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:43.020774 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:43.223124 [ 3048 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:43.438565 [ 3055 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:43.661468 [ 3061 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:43.735836 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:43.951647 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:43.981190 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:44.227388 [ 3072 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:44.441313 [ 3071 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:44.626852 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:44.654215 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:44.663899 [ 3085 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:44.863543 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:45.230451 [ 3082 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:45.449036 [ 3106 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:45.449800 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:45.578142 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:45.646446 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:45.666739 [ 3077 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:45.911985 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:46.237973 [ 3131 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:46.308610 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:46.373061 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:46.373147 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:46.451944 [ 3130 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:46.532200 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:46.669107 [ 3137 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:46.726556 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:46.754758 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:47.241152 [ 3153 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:47.263535 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:47.455052 [ 3156 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:47.675417 [ 3177 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:47.842846 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:47.852940 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:48.178759 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:48.243325 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:48.244116 [ 3182 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:48.389860 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:48.458208 [ 3194 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:48.679043 [ 3197 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:49.017000 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:49.030009 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:49.250478 [ 3210 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:49.423659 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:49.460456 [ 3204 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:49.681374 [ 3219 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:49.868414 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:50.246185 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:50.252792 [ 3237 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:50.254976 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:50.462364 [ 3249 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:50.683411 [ 3242 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:50.874279 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:50.908730 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:51.051822 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:51.254754 [ 3257 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:51.319486 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:51.464241 [ 2756 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:51.693792 [ 2754 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:51.783985 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:51.796648 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:52.171520 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:52.218305 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:52.257000 [ 2789 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:52.295778 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:52.471172 [ 2785 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:52.494253 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:52.509050 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:52.696764 [ 2787 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:53.040002 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:53.189320 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:53.260237 [ 2801 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:53.473818 [ 2827 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:53.611566 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:53.645162 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:53.699809 [ 2828 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:53.953491 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:54.263470 [ 2821 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:54.476888 [ 2836 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:54.667844 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:54.698777 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:54.702863 [ 2838 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:55.037025 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:55.265900 [ 2852 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:55.479704 [ 2853 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:55.705497 [ 2873 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:55.717154 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:55.745047 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:55.971301 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:56.215400 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:56.267818 [ 2876 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:56.482543 [ 2892 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:56.551106 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:56.559650 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:56.707726 [ 2897 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:57.112607 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:57.122488 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:57.131730 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:57.270295 [ 2916 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:57.294130 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:57.484658 [ 2907 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:57.644679 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:57.663214 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:57.710786 [ 2923 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:57.956550 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:58.031999 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:58.099288 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:58.273409 [ 2934 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:58.487930 [ 2940 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:58.624647 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:58.634859 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:58.713850 [ 2936 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:58.736677 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:58.894143 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:58.999613 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:59.233738 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:59.244370 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:59.261656 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:59.276744 [ 2960 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:59.495336 [ 2987 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:33:59.717249 [ 2976 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:00.144684 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:00.279350 [ 2992 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:00.497928 [ 2993 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:00.608440 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:00.636606 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:00.720434 [ 2999 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:00.973934 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:01.281388 [ 3012 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:01.435357 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:01.501105 [ 3009 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:01.535407 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:01.722857 [ 3035 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:02.071453 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:02.274951 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:02.283547 [ 3042 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:02.504326 [ 3046 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:02.594505 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:02.722389 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:02.725584 [ 3048 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:02.986584 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:03.217160 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:03.260581 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:03.287659 [ 3080 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:03.293738 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:03.507790 [ 3073 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:03.728444 [ 3091 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:03.814666 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:03.827671 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:04.290639 [ 3082 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:04.335128 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:04.367731 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:04.388637 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:04.520280 [ 3104 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:04.731467 [ 3123 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:05.006322 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:05.129471 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:05.319443 [ 3128 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:05.358368 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:05.523896 [ 3151 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:05.735287 [ 3133 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:06.139310 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:06.175667 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:06.364552 [ 3144 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:06.408081 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:06.456933 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:06.482995 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:06.527369 [ 3171 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:06.738959 [ 3172 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:06.817865 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:07.368486 [ 3180 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:07.464879 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:07.531554 [ 3200 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:07.686950 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:07.739481 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:07.742355 [ 3179 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:08.072615 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:08.460513 [ 3217 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:08.534235 [ 3190 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:08.600800 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:08.628582 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:08.750257 [ 3225 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:09.171028 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:09.463863 [ 3246 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:09.497614 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:09.536472 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:09.537031 [ 3235 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:09.766547 [ 3256 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:10.247175 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:10.496084 [ 2752 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:10.497324 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:10.540741 [ 2764 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:10.550736 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:10.769577 [ 2767 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:11.272988 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:11.400537 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:11.429596 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:11.508743 [ 2778 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:11.544378 [ 2751 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:11.597696 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:11.772892 [ 2787 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:11.895132 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:12.425699 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:12.436162 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:12.551318 [ 2800 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:12.552162 [ 2815 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:12.748778 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:12.763897 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:12.776261 [ 2806 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:13.025044 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:13.204777 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:13.278024 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:13.293550 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:13.558598 [ 2844 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:13.559025 [ 2823 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:13.614906 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:13.660355 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:13.691759 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:13.779698 [ 2860 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:14.392697 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:14.439081 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:14.562409 [ 2871 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:14.650065 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:14.658138 [ 2870 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:14.783162 [ 2876 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:14.806549 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:14.818950 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:15.289153 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:15.565775 [ 2901 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:15.661088 [ 2882 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:15.786291 [ 2891 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:16.000322 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:16.042447 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:16.124149 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:16.173234 [ 3354 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {fffcce7c-f4e8-42ed-b1c9-22aa3ca82e93::202509_245_250_1}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001adfc17a 4. ./build_docker/./src/Common/MemoryTracker.cpp:187: MemoryTracker::injectFault() const @ 0x000000001adf8d1a 5. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x000000002cc30a65 6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f47b 7. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:16.568025 [ 2910 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:16.663462 [ 2921 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:16.788586 [ 2923 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:16.965321 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:16.971488 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:17.117118 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:17.569958 [ 2930 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:17.665457 [ 2950 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:17.790712 [ 2954 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:17.971331 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:17.999770 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:18.217290 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:18.265491 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:18.272601 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:18.571827 [ 2971 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:18.667564 [ 2978 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:18.792465 [ 2972 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:18.807766 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:19.100009 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:19.113297 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:19.248917 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:19.573931 [ 2933 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:19.669882 [ 3017 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:19.795784 [ 3012 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:19.861626 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:19.897583 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:20.229111 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:20.576846 [ 3041 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:20.631670 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:20.645224 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:20.673325 [ 3029 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:20.799245 [ 3039 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:20.897384 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:21.579179 [ 3058 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:21.675497 [ 3067 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:21.714803 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:21.755936 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:21.767266 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:21.802589 [ 3069 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:22.042793 [ 3816 ] {e895af0c-8c9b-4a42-a779-e091ae5a92e8} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:33010) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:34:22.048880 [ 3816 ] {e895af0c-8c9b-4a42-a779-e091ae5a92e8} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:34:22.581885 [ 3089 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:22.678160 [ 3099 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:22.804923 [ 3102 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:22.819641 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:22.902047 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:22.914747 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:23.584654 [ 3119 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:23.617053 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:23.681286 [ 3134 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:23.753978 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:23.790145 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:23.807527 [ 3129 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:24.042185 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:24.067394 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:24.281730 [ 3355 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {02260661-95d0-4eca-a73d-2543d99ce5b9::202509_388_443_11}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001adfc17a 4. ./build_docker/./src/Common/MemoryTracker.cpp:187: MemoryTracker::injectFault() const @ 0x000000001adf8d1a 5. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x000000002cc30a65 6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f47b 7. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:24.356863 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:24.377781 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:24.423300 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:24.588014 [ 3154 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:24.691060 [ 3158 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:24.716944 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:24.735815 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:24.811345 [ 3143 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:25.123213 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:25.300147 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:25.434453 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:25.445644 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:25.592676 [ 3184 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:25.681900 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:25.694191 [ 3097 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:25.703555 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:25.763077 [ 3816 ] {690e17dd-b36e-4844-a1f7-ad7511dd49f4} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:33038) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:34:25.767516 [ 3816 ] {690e17dd-b36e-4844-a1f7-ad7511dd49f4} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:34:25.820651 [ 3188 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:25.954280 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:26.179601 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:26.200906 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:26.596311 [ 3209 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:26.707003 [ 3208 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:26.712406 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:26.824189 [ 3213 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:27.412019 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:27.445162 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:27.598740 [ 3238 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:27.622533 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:27.709792 [ 3249 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:27.826985 [ 3247 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:28.389543 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:28.411219 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:28.530843 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:28.601307 [ 2759 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:28.712783 [ 2756 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:28.830300 [ 2762 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:29.060666 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:29.085189 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:29.170749 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:29.279097 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:29.604925 [ 2770 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:29.709538 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:29.715597 [ 2786 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:29.735211 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:29.834303 [ 2785 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:30.414208 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:30.607975 [ 2812 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:30.724469 [ 2796 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:30.789940 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:30.800861 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:30.842745 [ 2805 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:31.222823 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:31.247910 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:31.260193 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:31.610665 [ 2848 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:31.636957 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:31.727700 [ 2838 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:31.846370 [ 2823 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:31.848396 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:31.863911 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:32.378458 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:32.391681 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:32.613874 [ 2864 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:32.731419 [ 2868 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:32.768926 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:32.849954 [ 2878 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:33.122816 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:33.167736 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:33.263730 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:33.616885 [ 2883 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:33.734579 [ 2896 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:33.853060 [ 2901 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:34.052794 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:34.395407 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:34.470286 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:34.636828 [ 2910 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:34.732445 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:34.737414 [ 2922 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:34.743514 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:34.856219 [ 2909 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:35.167076 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:35.405329 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:35.447609 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:35.639247 [ 2947 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:35.740264 [ 2955 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:35.781261 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:35.810987 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:35.859101 [ 2959 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:36.238944 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:36.469700 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:36.642145 [ 2970 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:36.707763 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:36.737161 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:36.743393 [ 2989 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:36.862194 [ 2994 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:37.309388 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:37.644059 [ 3002 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:37.745736 [ 3010 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:37.805287 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:37.817774 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:37.864850 [ 3003 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:38.338931 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:38.410359 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:38.435326 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:38.651893 [ 3036 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:38.748384 [ 3042 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:38.868792 [ 3038 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:39.131425 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:39.139548 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:39.175849 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:39.584345 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:39.655162 [ 3064 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:39.751944 [ 3054 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:39.872224 [ 3067 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:40.124199 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:40.135063 [ 3816 ] {952e7222-dea9-4e31-95f7-0e862da42739} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43096) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:34:40.135451 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:40.139441 [ 3816 ] {952e7222-dea9-4e31-95f7-0e862da42739} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=0): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82a56b 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:34:40.580441 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:40.592267 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:40.658531 [ 3081 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:40.659482 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:40.754985 [ 3087 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:40.923120 [ 3092 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:41.220901 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:41.706951 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:41.661989 [ 3122 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:41.758081 [ 3111 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:41.760688 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:41.859925 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:41.926374 [ 3119 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:42.344805 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:42.532175 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:42.620143 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:42.641424 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:42.744247 [ 3140 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:42.764078 [ 3149 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:42.823841 [ 3359 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {f2eeeeb5-70de-4a33-835e-3bff489e4be8::202509_1_203_99}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001adfc17a 4. ./build_docker/./src/Common/MemoryTracker.cpp:187: MemoryTracker::injectFault() const @ 0x000000001adf8d1a 5. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x000000002cc30a65 6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f47b 7. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:42.928791 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:42.929300 [ 3144 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:43.149758 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:43.158538 [ 3355 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:43.462040 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:43.504450 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:43.747693 [ 3159 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:43.767078 [ 3184 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:43.952627 [ 3175 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:44.107586 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:44.494495 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:44.533595 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:44.578481 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:44.751470 [ 3169 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:44.770634 [ 3201 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:44.956155 [ 3205 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:45.429652 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:45.739155 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:45.754444 [ 3226 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:45.773697 [ 3244 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:45.864378 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:45.959509 [ 3237 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:46.345820 [ 3816 ] {e7e75331-d9c8-4337-8a66-1c8ba47091d5} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:43130) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:34:46.354747 [ 3816 ] {e7e75331-d9c8-4337-8a66-1c8ba47091d5} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=15, jobs=1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeReadTask.cpp:44: DB::MergeTreeReadTask::createReaders(std::shared_ptr const&, DB::MergeTreeReadTask::Extras const&, DB::MarkRanges const&) @ 0x000000002c82acda 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:295: DB::MergeTreeReadPoolBase::createTask(std::shared_ptr, DB::MarkRanges, DB::MergeTreeReadTask*) const @ 0x000000002de9c234 14. ./build_docker/./src/Storages/MergeTree/MergeTreeReadPoolInOrder.cpp:72: DB::MergeTreeReadPoolInOrder::getTask(unsigned long, DB::MergeTreeReadTask*) @ 0x000000002deff671 15. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectAlgorithms.cpp:17: DB::MergeTreeInOrderSelectAlgorithm::getNewTask(DB::IMergeTreeReadPool&, DB::MergeTreeReadTask*) @ 0x000000002dea3919 16. ./build_docker/./src/Storages/MergeTree/MergeTreeSelectProcessor.cpp:0: DB::MergeTreeSelectProcessor::read() @ 0x000000002c8241d2 17. ./build_docker/./src/Storages/MergeTree/MergeTreeSource.cpp:180: DB::MergeTreeSource::tryGenerate() @ 0x000000002de6d77a 18. ./build_docker/./src/Processors/ISource.cpp:0: DB::ISource::work() @ 0x000000002d3290a1 19. ./build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:0: DB::ExecutionThreadContext::executeTask() @ 0x000000002d362fee 20. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x000000002d349dd1 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d34877c 22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 23. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007f12cd75eac3 28. ? @ 0x00007f12cd7f0850 2025.09.25 21:34:46.532669 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:46.776499 [ 3255 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:46.757977 [ 3258 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:46.787504 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:46.826191 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:46.963052 [ 3260 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:47.099608 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:47.249266 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:47.779985 [ 2773 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:47.784331 [ 2748 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:47.883807 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:47.966790 [ 2774 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:48.061470 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:48.072908 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:48.486050 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:48.783044 [ 2787 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:48.787182 [ 2794 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:48.811727 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:48.899399 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:48.969127 [ 2790 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:49.152835 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:49.376489 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:49.553869 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:49.785779 [ 2830 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:49.789848 [ 2822 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:49.943241 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:49.972094 [ 2819 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:49.975869 [ 3354 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:50.532822 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:50.788610 [ 2845 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:50.792758 [ 2857 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:50.983514 [ 2852 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:50.988429 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:51.001356 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:51.438484 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:51.642247 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:51.748704 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:51.790615 [ 2874 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:51.798962 [ 2875 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:51.986935 [ 2869 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:52.469232 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:52.609485 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:52.637821 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:52.804281 [ 2904 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:52.805210 [ 2902 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:52.998523 [ 2916 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:53.498245 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:53.707378 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:53.771730 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:53.791129 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:53.811663 [ 2918 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:53.812289 [ 2932 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:54.006136 [ 2927 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:54.731380 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:54.815966 [ 2954 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:54.818169 [ 2951 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:54.828812 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:54.876427 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:55.013605 [ 2955 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:55.647797 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:55.819673 [ 2969 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:55.821888 [ 2957 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:56.001962 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:56.021241 [ 2967 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:56.054823 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:56.105548 [ 3816 ] {c411ab1c-2427-4770-9664-f4b3a2277bf4} executeQuery: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::1]:9000 -> [::1]:48514). (NETWORK_ERROR) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:48514) (comment: 02585_query_status_deadlock.sh) (in query: select query, event_time from system.query_log where query_id = 'test_5_test_02585_query_to_kill_id_1' and current_database = 'test_5' and query like 'select%' limit 1), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001b0bab02 4. ./build_docker/./src/IO/WriteBufferFromPocoSocketChunked.cpp:201: DB::WriteBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf09bf7 5. ./src/IO/WriteBuffer.h:66: DB::TCPHandler::sendProfileInfo(DB::ProfileInfo const&) @ 0x000000002d196a37 6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18b607 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:34:56.117707 [ 3816 ] {c411ab1c-2427-4770-9664-f4b3a2277bf4} TCPHandler: Can't send logs to client: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::1]:9000 -> [::1]:48514). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:169: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001b0bada2 4. ./build_docker/./src/IO/WriteBufferFromPocoSocketChunked.cpp:201: DB::WriteBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf09bf7 5. ./src/IO/WriteBuffer.h:66: DB::TCPHandler::sendLogData(DB::Block const&) @ 0x000000002d1a9beb 6. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::TCPHandler::sendLogs() @ 0x000000002d18db59 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d176902 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:56.120329 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:56.120546 [ 3816 ] {c411ab1c-2427-4770-9664-f4b3a2277bf4} TCPHandler: Code: 210. DB::NetException: Connection reset by peer, while writing to socket ([::1]:9000 -> [::1]:48514). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. ./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000001b0b7a5f 3. ./build_docker/./src/IO/WriteBufferFromPocoSocket.cpp:158: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x000000001b0bab02 4. ./build_docker/./src/IO/WriteBufferFromPocoSocketChunked.cpp:201: DB::WriteBufferFromPocoSocketChunked::nextImpl() @ 0x000000002cf09bf7 5. ./src/IO/WriteBuffer.h:66: DB::TCPHandler::sendProfileInfo(DB::ProfileInfo const&) @ 0x000000002d196a37 6. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQuery() @ 0x000000002d18b607 7. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x000000002d17644a 8. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1a9fa0 9. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 11. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 12. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 2025.09.25 21:34:56.123747 [ 3816 ] {c411ab1c-2427-4770-9664-f4b3a2277bf4} TCPHandler: Client has gone away. 2025.09.25 21:34:56.382613 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:56.758641 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:56.770372 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:56.793114 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:56.822625 [ 2999 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:56.824798 [ 2995 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:57.024630 [ 3010 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:57.120479 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:57.155903 [ 3357 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:57.240568 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:57.825521 [ 3032 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:57.828224 [ 3036 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:57.921229 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:57.942509 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:58.012164 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:58.028604 [ 3038 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:58.753688 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:58.829527 [ 3064 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:58.831032 [ 3055 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:58.871162 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:58.940856 [ 3350 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:59.031109 [ 3060 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:59.487731 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:59.832586 [ 3089 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:59.840505 [ 3079 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:59.914468 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:34:59.937958 [ 3353 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:00.029055 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:00.033367 [ 3086 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:00.180272 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:00.350169 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:00.744500 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:00.767328 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:00.836141 [ 3113 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:00.845998 [ 3109 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:01.036164 [ 3118 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:01.069255 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:01.357339 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:01.378996 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:01.838800 [ 3151 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:01.848513 [ 3130 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:02.000249 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:02.039140 [ 3137 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:02.409650 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:02.434762 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:02.682078 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:02.714206 [ 3362 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:02.813943 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:02.841909 [ 3161 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:02.851454 [ 3156 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:03.042296 [ 3160 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:03.414631 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:03.433915 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:03.445785 [ 3381 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:03.673598 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:03.688464 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:03.708963 [ 3383 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:03.845061 [ 3187 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:03.854893 [ 3196 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:04.044859 [ 3191 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:04.103678 [ 3384 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:04.188123 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:04.209272 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:04.474492 [ 3386 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:04.847916 [ 3213 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:04.860189 [ 3217 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:05.047858 [ 3190 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:05.141379 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:05.159936 [ 3356 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:05.399845 [ 3388 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:05.734674 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:05.762644 [ 3361 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:05.832740 [ 3374 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:05.851075 [ 3256 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:05.862892 [ 3250 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:06.050160 [ 2747 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:06.851302 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:06.853572 [ 2762 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:06.865563 [ 2761 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:06.869248 [ 3364 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:06.933323 [ 3376 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:07.053270 [ 2773 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:07.363334 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:07.395056 [ 3365 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:07.406014 [ 3378 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:07.716095 [ 3382 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:07.857061 [ 2758 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:07.871646 [ 2792 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:07.949015 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:08.005804 [ 3359 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:08.056594 [ 2790 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:08.716614 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:08.727746 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:08.836129 [ 3380 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:08.860005 [ 2806 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:08.874361 [ 2808 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:09.060255 [ 2802 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:09.644438 [ 3385 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:09.877478 [ 2838 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:09.958775 [ 2839 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:10.020525 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:10.061813 [ 3387 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:10.063667 [ 2823 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:10.064844 [ 3360 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:10.880763 [ 3389 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:10.974482 [ 2863 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:10.974622 [ 2862 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:11.060071 [ 3375 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:11.065117 [ 2924 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:11.068290 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:11.069681 [ 3352 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:11.387089 [ 3377 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:11.394297 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:11.398367 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:11.726458 [ 3379 ] {} test_3.rmt2 (d7bb0947-a3c4-42c8-aa9c-d77098a8f33b): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server http://fa9ed7cbd16c:9009/?endpoint=DataPartsExchange%3Adefault%3A%2Ftest%2F01165%2Ftest_3%2Frmt%2Freplicas%2F1&part=20250925_0_0_0&client_protocol_version=8&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 236. DB::Exception: Transferring part to replica was cancelled. (ABORTED) (version 24.8.14.10508.altinitytest (altinity build)). (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::HTTPException::makeExceptionMessage(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bceec 4. ./src/Common/Exception.h:43: DB::HTTPException::HTTPException(int, String const&, Poco::Net::HTTPResponse::HTTPStatus, String const&, String const&) @ 0x000000001b3bd234 5. ./build_docker/./src/IO/HTTPCommon.cpp:91: DB::assertResponseIsOk(String const&, Poco::Net::HTTPResponse&, std::basic_istream>&, bool) @ 0x000000001b3bcc31 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x000000001b534553 7. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x000000001b534a79 8. ./contrib/llvm-project/libcxx/include/optional:344: DB::ReadWriteBufferFromHTTP::initialize() @ 0x000000001b535c79 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void std::__function::__policy_invoker::__call_impl>(std::__function::__policy_storage const*) @ 0x000000001b53b418 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b530531 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001b5383e3 12. DB::ReadBuffer::next() @ 0x000000000b528aeb 13. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:0: DB::ReadWriteBufferFromHTTP::ReadWriteBufferFromHTTP(DB::HTTPConnectionGroupType const&, Poco::URI const&, String const&, DB::ProxyConfiguration, DB::ReadSettings, DB::ConnectionTimeouts, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*, unsigned long, unsigned long, std::function>&)>, bool, bool, std::vector>, bool, std::optional) @ 0x000000001b53359d 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if::__unique_single std::make_unique[abi:v15007]>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&>(DB::HTTPConnectionGroupType&, Poco::URI&, String&, DB::ProxyConfiguration&, DB::ReadSettings&, DB::ConnectionTimeouts&, Poco::Net::HTTPBasicCredentials const&, DB::RemoteHostFilter const*&, unsigned long&, unsigned long&, std::function>&)>&, bool&, bool&, std::vector>&, bool&, std::nullopt_t const&) @ 0x000000001b5263f5 15. ./src/IO/ReadWriteBufferFromHTTP.h:242: DB::DataPartsExchange::Fetcher::fetchSelectedPart(std::shared_ptr const&, std::shared_ptr, String const&, String const&, String const&, String const&, int, DB::ConnectionTimeouts const&, String const&, String const&, String const&, std::shared_ptr, bool, String const&, std::optional*, bool, std::shared_ptr) @ 0x000000002c2404fc 16. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:4941: std::shared_ptr std::__function::__policy_invoker ()>::__call_impl const&, String const&, String const&, bool, unsigned long, std::shared_ptr, bool)::$_4, std::shared_ptr ()>>(std::__function::__policy_storage const*) @ 0x000000002bc26ed2 17. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000002bab850c 18. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2297: DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x000000002baa2c79 19. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:2137: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x000000002ba7eb0a 20. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002bc21a5a 21. ./build_docker/./src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp:1817: DB::ReplicatedMergeTreeQueue::processEntry(std::function ()>, std::shared_ptr&, std::function&)>) @ 0x000000002caa71e5 22. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr) @ 0x000000002baf788f 23. ./src/Storages/MergeTree/IExecutableTask.h:70: DB::ExecutableLambdaAdapter::executeStep() @ 0x000000002bc77054 24. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c421856 25. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c422dc5 26. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 27. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 28. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 29. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 30. asan_thread_start(void*) @ 0x000000000aa43059 31. ? @ 0x00007f12cd75eac3 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:11.975009 [ 3022 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:11.975343 [ 3003 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:12.035905 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:12.039582 [ 3363 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:12.065924 [ 3026 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:12.975568 [ 3084 ] {} rdb_test_01iwczf0_4.rmt (96b435d1-cac3-40e7-b94b-e2a8d173cac2): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/96b435d1-cac3-40e7-b94b-e2a8d173cac2/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:12.975889 [ 3079 ] {} rdb_test_6nj3uy8j_4.rmt (459472c2-9ab3-438b-a1dc-068548edaa54): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/459472c2-9ab3-438b-a1dc-068548edaa54/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:13.054744 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:13.056489 [ 3358 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:13.066487 [ 3087 ] {} rdb_test_knhvna7w_4.rmt (4835d234-2c5d-4982-b760-6c9442b1ea5f): void DB::StorageReplicatedMergeTree::mutationsFinalizingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/4835d234-2c5d-4982-b760-6c9442b1ea5f/s1/replicas/r1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x000000000aabb3f4 4. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0x000000002ba11483 5. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:262: DB::StorageReplicatedMergeTree::mutationsFinalizingTask() @ 0x000000002bafe8f1 6. ./src/Common/Stopwatch.h:72: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000271eb2d0 7. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000271f11dd 8. ./src/Common/ThreadPool.h:312: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl, StrongTypedef, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef, StrongTypedef, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000271f2387 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 10. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 11. asan_thread_start(void*) @ 0x000000000aa43059 12. ? @ 0x00007f12cd75eac3 13. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:13.388068 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:13.400034 [ 3351 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007f12cd75eac3 20. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:35:52.677553 [ 3359 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {261a6bdf-2436-455f-b6df-dbd9b591b8be::202509_1_244_127}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001adfc17a 4. ./build_docker/./src/Common/MemoryTracker.cpp:187: MemoryTracker::injectFault() const @ 0x000000001adf8d1a 5. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::finish() @ 0x000000002cc30a65 6. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f47b 7. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 10. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 11. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 12. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 13. asan_thread_start(void*) @ 0x000000000aa43059 14. ? @ 0x00007f12cd75eac3 15. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:13.845574 [ 19210 ] {b26efec4-63a7-4cb0-a9c0-9f5f7b5bb724} executeQuery: Code: 159. DB::Exception: Timeout exceeded: elapsed 60.048637494 seconds, maximum: 60. (TIMEOUT_EXCEEDED) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:42878) (in query: SELECT sleepEachRow(( SELECT maxOrDefault(300 - elapsed) + 1 FROM system.processes WHERE query NOT LIKE '%FROM system.processes%' AND elapsed < 300 ) / 300) FROM numbers(300) FORMAT Null SETTINGS function_sleep_max_microseconds_per_block = 0 ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002785801a 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000027856784 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:36:13.846031 [ 19210 ] {b26efec4-63a7-4cb0-a9c0-9f5f7b5bb724} TCPHandler: Code: 159. DB::Exception: Timeout exceeded: elapsed 60.048637494 seconds, maximum: 60. (TIMEOUT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, double&&, double&&) @ 0x000000002785801a 4. ./build_docker/./src/QueryPipeline/ExecutionSpeedLimits.cpp:111: DB::ExecutionSpeedLimits::checkTimeLimit(Stopwatch const&, DB::OverflowMode) const @ 0x0000000027856784 5. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:0: DB::PipelineExecutor::finalizeExecution() @ 0x000000002d3489b5 6. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:132: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d348340 7. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007f12cd75eac3 12. ? @ 0x00007f12cd7f0850 2025.09.25 21:36:14.585386 [ 13110 ] {f63bb9c4-b4c0-4bf0-8f54-9f6afe1badd8} executeQuery: Code: 521. DB::ErrnoException: Cannot rename /var/lib/clickhouse/metadata/test.sql.tmp to /var/lib/clickhouse/metadata/test.sql because the second path already exists: , errno: 17, strerror: File exists. (ATOMIC_RENAME_FAIL) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:57694) (in query: CREATE DATABASE IF NOT EXISTS test ), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::ErrnoException::ErrnoException(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String const&) @ 0x0000000027d0c015 4. ./build_docker/./src/Common/atomicRename.cpp:92: DB::renameat2(String const&, String const&, int) @ 0x0000000027d0bc1b 5. ./build_docker/./src/Common/atomicRename.cpp:213: DB::renameNoReplace(String const&, String const&) @ 0x0000000027d0b399 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x000000002918eb44 7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c33d8 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 10. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 11. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 2025.09.25 21:36:14.586651 [ 13110 ] {f63bb9c4-b4c0-4bf0-8f54-9f6afe1badd8} DynamicQueryHandler: Code: 521. DB::ErrnoException: Cannot rename /var/lib/clickhouse/metadata/test.sql.tmp to /var/lib/clickhouse/metadata/test.sql because the second path already exists: , errno: 17, strerror: File exists. (ATOMIC_RENAME_FAIL), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::ErrnoException::ErrnoException(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String const&) @ 0x0000000027d0c015 4. ./build_docker/./src/Common/atomicRename.cpp:92: DB::renameat2(String const&, String const&, int) @ 0x0000000027d0bc1b 5. ./build_docker/./src/Common/atomicRename.cpp:213: DB::renameNoReplace(String const&, String const&) @ 0x0000000027d0b399 6. ./contrib/llvm-project/libcxx/include/string:1499: DB::InterpreterCreateQuery::createDatabase(DB::ASTCreateQuery&) @ 0x000000002918eb44 7. ./build_docker/./src/Interpreters/InterpreterCreateQuery.cpp:1986: DB::InterpreterCreateQuery::execute() @ 0x00000000291c33d8 8. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b382fc 9. ./build_docker/./src/Interpreters/executeQuery.cpp:1529: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x0000000029b3ed3f 10. ./build_docker/./src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002d0662d1 11. ./build_docker/./src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002d06d139 12. ./build_docker/./src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x000000002d1cc337 13. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x0000000034530bef 14. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x00000000345317d7 15. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x0000000034433eeb 16. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003442e048 17. asan_thread_start(void*) @ 0x000000000aa43059 18. ? @ 0x00007f12cd75eac3 19. ? @ 0x00007f12cd7f0850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:21.072387 [ 2702 ] {} Application: Closed all listening sockets. Waiting for 13 outstanding connections. 2025.09.25 21:36:25.978665 [ 2702 ] {} Application: Closed connections. But 13 remain. Tip: To increase wait time add to config: 60 2025.09.25 21:36:25.978930 [ 2702 ] {} Application: Will shutdown forcefully. 2025.09.25 21:36:27.648292 [ 19277 ] {} Context: Server was built with address sanitizer. It will work slowly. 2025.09.25 21:36:27.662323 [ 19277 ] {} Context: Server logging level is set to 'test' and performance is degraded. This cannot be used in production. 2025.09.25 21:36:27.663002 [ 19277 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. 2025.09.25 21:36:27.663503 [ 19277 ] {} Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. 2025.09.25 21:36:28.763368 [ 19277 ] {} KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647. 2025.09.25 21:36:28.774936 [ 19277 ] {} RaftInstance: invalid election timeout lower bound detected, adjusted to 0 2025.09.25 21:36:28.774983 [ 19277 ] {} RaftInstance: invalid election timeout upper bound detected, adjusted to 0 2025.09.25 21:36:28.826227 [ 19868 ] {} RaftInstance: Election timeout, initiate leader election 2025.09.25 21:36:30.469001 [ 19277 ] {} Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring. 2025.09.25 21:36:31.005059 [ 19277 ] {} loadMetadata: Removing temporary file /var/lib/clickhouse/metadata/test.sql.tmp 2025.09.25 21:36:31.087144 [ 19277 ] {} DatabaseReplicated (rdb_test_01iwczf0): Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/test_01iwczf0/rdb already exists. Replica host ID: 'fa9ed7cbd16c:9000:4511ad98-808f-4623-a415-6728d87576ec', current host ID: 'fa9ed7cbd16c:9000:642f182e-aae8-4e08-a0b3-b9c9a82e7581'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, String&, String&, String&, String&, String&) @ 0x0000000027d8bad5 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:458: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000027d43914 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr, DB::LoadingStrictnessLevel) @ 0x0000000027d4bcb1 6. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set, std::hash>, std::equal_to>, std::allocator>>) @ 0x00000000291db222 7. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::loadMetadata(std::shared_ptr, String const&, bool) @ 0x0000000029b7c396 8. ./contrib/llvm-project/libcxx/include/vector:961: DB::Server::main(std::vector> const&) @ 0x000000001b16b8ff 9. ./build_docker/./base/poco/Util/src/Application.cpp:0: Poco::Util::Application::run() @ 0x0000000034593cf7 10. ./build_docker/./programs/server/Server.cpp:396: DB::Server::run() @ 0x000000001b14b97a 11. ./build_docker/./programs/server/Server.cpp:0: mainEntryClickHouseServer(int, char**) @ 0x000000001b146de2 12. ./build_docker/./programs/main.cpp:0: main @ 0x000000000aa7c909 13. ? @ 0x00007fa4c63a7d90 14. ? @ 0x00007fa4c63a7e40 15. _start @ 0x000000000a9ad02e (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:31.127731 [ 19277 ] {} DatabaseReplicated (rdb_test_6nj3uy8j): Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/test_6nj3uy8j/rdb already exists. Replica host ID: 'fa9ed7cbd16c:9000:6c6032c7-b6dc-41ae-8b79-3099020372a2', current host ID: 'fa9ed7cbd16c:9000:8e9bbf41-e59a-447b-bd19-90e784045e46'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, String&, String&, String&, String&, String&) @ 0x0000000027d8bad5 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:458: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000027d43914 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr, DB::LoadingStrictnessLevel) @ 0x0000000027d4bcb1 6. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set, std::hash>, std::equal_to>, std::allocator>>) @ 0x00000000291db222 7. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::loadMetadata(std::shared_ptr, String const&, bool) @ 0x0000000029b7c396 8. ./contrib/llvm-project/libcxx/include/vector:961: DB::Server::main(std::vector> const&) @ 0x000000001b16b8ff 9. ./build_docker/./base/poco/Util/src/Application.cpp:0: Poco::Util::Application::run() @ 0x0000000034593cf7 10. ./build_docker/./programs/server/Server.cpp:396: DB::Server::run() @ 0x000000001b14b97a 11. ./build_docker/./programs/server/Server.cpp:0: mainEntryClickHouseServer(int, char**) @ 0x000000001b146de2 12. ./build_docker/./programs/main.cpp:0: main @ 0x000000000aa7c909 13. ? @ 0x00007fa4c63a7d90 14. ? @ 0x00007fa4c63a7e40 15. _start @ 0x000000000a9ad02e (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:31.140678 [ 19277 ] {} DatabaseReplicated (rdb_test_knhvna7w): Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/test_knhvna7w/rdb already exists. Replica host ID: 'fa9ed7cbd16c:9000:e021de81-635c-4962-8732-ca74e4a7f0cb', current host ID: 'fa9ed7cbd16c:9000:901fbf11-3c89-46c1-ba8d-27b10826f797'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, String&, String&, String&, String&, String&) @ 0x0000000027d8bad5 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:458: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000027d43914 5. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::DatabaseReplicated::beforeLoadingMetadata(std::shared_ptr, DB::LoadingStrictnessLevel) @ 0x0000000027d4bcb1 6. ./build_docker/./src/Databases/TablesLoader.cpp:0: DB::TablesLoader::loadTablesAsync(std::unordered_set, std::hash>, std::equal_to>, std::allocator>>) @ 0x00000000291db222 7. ./contrib/llvm-project/libcxx/include/__hash_table:1473: DB::loadMetadata(std::shared_ptr, String const&, bool) @ 0x0000000029b7c396 8. ./contrib/llvm-project/libcxx/include/vector:961: DB::Server::main(std::vector> const&) @ 0x000000001b16b8ff 9. ./build_docker/./base/poco/Util/src/Application.cpp:0: Poco::Util::Application::run() @ 0x0000000034593cf7 10. ./build_docker/./programs/server/Server.cpp:396: DB::Server::run() @ 0x000000001b14b97a 11. ./build_docker/./programs/server/Server.cpp:0: mainEntryClickHouseServer(int, char**) @ 0x000000001b146de2 12. ./build_docker/./programs/main.cpp:0: main @ 0x000000000aa7c909 13. ? @ 0x00007fa4c63a7d90 14. ? @ 0x00007fa4c63a7e40 15. _start @ 0x000000000a9ad02e (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:31.151990 [ 19277 ] {} DatabaseReplicated (rdb_test_t1fje8px): Database rdb_test_t1fje8px exists locally, but its replica does not exist in ZooKeeper (/test/test_t1fje8px/rdb/replicas/s1|r1). Assuming it was dropped, will not try to start it up 2025.09.25 21:36:31.221827 [ 19277 ] {} TablesLoader: Tables test_5.dist_01247 depend on default.number, but seems like that does not exist. Will ignore it and try to load existing tables 2025.09.25 21:36:31.393987 [ 19894 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:31.409080 [ 20000 ] {} DDLWorker(rdb_test_01iwczf0): Error on initialization of rdb_test_01iwczf0: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/test_01iwczf0/rdb already exists. Replica host ID: 'fa9ed7cbd16c:9000:4511ad98-808f-4623-a415-6728d87576ec', current host ID: 'fa9ed7cbd16c:9000:642f182e-aae8-4e08-a0b3-b9c9a82e7581'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, String&, String&, String&, String&, String&) @ 0x0000000027d8bad5 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:458: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000027d43914 5. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9c95d 6. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 7. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007fa4c6412ac3 12. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:31.409148 [ 19958 ] {} DDLWorker(rdb_test_knhvna7w): Error on initialization of rdb_test_knhvna7w: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/test_knhvna7w/rdb already exists. Replica host ID: 'fa9ed7cbd16c:9000:e021de81-635c-4962-8732-ca74e4a7f0cb', current host ID: 'fa9ed7cbd16c:9000:901fbf11-3c89-46c1-ba8d-27b10826f797'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, String&, String&, String&, String&, String&) @ 0x0000000027d8bad5 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:458: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000027d43914 5. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9c95d 6. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 7. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007fa4c6412ac3 12. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:31.409320 [ 19986 ] {} DDLWorker(rdb_test_6nj3uy8j): Error on initialization of rdb_test_6nj3uy8j: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/test_6nj3uy8j/rdb already exists. Replica host ID: 'fa9ed7cbd16c:9000:6c6032c7-b6dc-41ae-8b79-3099020372a2', current host ID: 'fa9ed7cbd16c:9000:8e9bbf41-e59a-447b-bd19-90e784045e46'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, String&, String&, String&, String&, String&) @ 0x0000000027d8bad5 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:458: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000027d43914 5. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9c95d 6. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 7. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007fa4c6412ac3 12. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:31.429313 [ 19894 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:31.464585 [ 20015 ] {} void DB::AsyncLoader::worker(Pool &): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=734, jobs=716). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af872f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af7b2d1 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af7c3f3 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002869ffdd 7. ./build_docker/./src/Interpreters/DDLWorker.cpp:132: DB::DDLWorker::startup() @ 0x0000000028671c40 8. ./build_docker/./src/Databases/DatabaseReplicated.cpp:0: void std::__function::__policy_invoker const&)>::__call_impl, std::hash>, std::equal_to>, std::allocator>>, DB::LoadingStrictnessLevel)::$_0, void (DB::AsyncLoader&, std::shared_ptr const&)>>(std::__function::__policy_storage const*, DB::AsyncLoader&, std::shared_ptr const&) @ 0x0000000027d89063 9. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001b223e7b 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 11. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 12. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 13. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 14. asan_thread_start(void*) @ 0x000000000aa43059 15. ? @ 0x00007fa4c6412ac3 16. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:32.106815 [ 19282 ] {cdab7503-4d0f-4a2c-9528-6a4a0878099e} executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK) (version 24.8.14.10508.altinitytest (altinity build)) (from [::1]:55508) (in query: SELECT count() FROM system.build_options WHERE name = 'CXX_FLAGS' AND position('sanitize=address' IN value)), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007fa4c6412ac3 14. ? @ 0x00007fa4c64a4850 2025.09.25 21:36:32.107318 [ 19282 ] {cdab7503-4d0f-4a2c-9528-6a4a0878099e} TCPHandler: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=7, jobs=7). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002d34b5ca 7. ./build_docker/./src/Processors/Executors/PipelineExecutor.cpp:405: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x000000002d3486e4 8. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:274: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x000000002d34827b 9. ./build_docker/./src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:94: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000002d36bcf7 10. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 12. asan_thread_start(void*) @ 0x000000000aa43059 13. ? @ 0x00007fa4c6412ac3 14. ? @ 0x00007fa4c64a4850 2025.09.25 21:36:32.561150 [ 19898 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:32.563372 [ 19898 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:33.372857 [ 19907 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:33.376403 [ 19907 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:34.268190 [ 19897 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:34.271306 [ 19897 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:35.225002 [ 19905 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:35.234338 [ 19905 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:36.174641 [ 19896 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:36.176988 [ 19896 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:36.411055 [ 20000 ] {} DDLWorker(rdb_test_01iwczf0): Error on initialization of rdb_test_01iwczf0: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/test_01iwczf0/rdb already exists. Replica host ID: 'fa9ed7cbd16c:9000:4511ad98-808f-4623-a415-6728d87576ec', current host ID: 'fa9ed7cbd16c:9000:642f182e-aae8-4e08-a0b3-b9c9a82e7581'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, String&, String&, String&, String&, String&) @ 0x0000000027d8bad5 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:458: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000027d43914 5. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9c95d 6. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 7. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007fa4c6412ac3 12. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:36.411343 [ 19958 ] {} DDLWorker(rdb_test_knhvna7w): Error on initialization of rdb_test_knhvna7w: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/test_knhvna7w/rdb already exists. Replica host ID: 'fa9ed7cbd16c:9000:e021de81-635c-4962-8732-ca74e4a7f0cb', current host ID: 'fa9ed7cbd16c:9000:901fbf11-3c89-46c1-ba8d-27b10826f797'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, String&, String&, String&, String&, String&) @ 0x0000000027d8bad5 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:458: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000027d43914 5. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9c95d 6. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 7. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007fa4c6412ac3 12. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:36.411455 [ 19986 ] {} DDLWorker(rdb_test_6nj3uy8j): Error on initialization of rdb_test_6nj3uy8j: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/test_6nj3uy8j/rdb already exists. Replica host ID: 'fa9ed7cbd16c:9000:6c6032c7-b6dc-41ae-8b79-3099020372a2', current host ID: 'fa9ed7cbd16c:9000:8e9bbf41-e59a-447b-bd19-90e784045e46'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, String&, String&, String&, String&, String&) @ 0x0000000027d8bad5 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:458: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000027d43914 5. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9c95d 6. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 7. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007fa4c6412ac3 12. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:36.704093 [ 19900 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:36.705893 [ 19900 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:37.126132 [ 19904 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:37.139462 [ 19904 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:38.040147 [ 19908 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {8b990b7c-a181-415f-9d90-25ec9d0f3599::202509_1_271_54}: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=3, jobs=0). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002c765aab 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:1197: ? @ 0x000000002c7602b8 8. ./build_docker/./src/Storages/MergeTree/MergeTreeMarksLoader.cpp:73: DB::MergeTreeMarksLoader::startAsyncLoad() @ 0x000000002c75fc2b 9. ./build_docker/./src/Storages/MergeTree/MergeTreeReaderCompact.cpp:0: DB::MergeTreeReaderCompact::MergeTreeReaderCompact(std::shared_ptr, DB::NamesAndTypesList, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*, DB::MarkCache*, DB::MarkRanges, DB::MergeTreeReaderSettings, std::map, std::allocator>>, std::function const&, int) @ 0x000000002c7aee03 10. ./src/Storages/MergeTree/MergeTreeReaderCompactSingleBuffer.h:16: DB::MergeTreeReaderCompactSingleBuffer::MergeTreeReaderCompactSingleBuffer&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int>(std::shared_ptr&, DB::NamesAndTypesList const&, std::unordered_map, std::equal_to, std::allocator>> const&, std::shared_ptr const&, DB::UncompressedCache*&, DB::MarkCache*&, DB::MarkRanges const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&, int&&) @ 0x000000002c5d6d21 11. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: DB::MergeTreeDataPartCompact::getReader(DB::NamesAndTypesList const&, std::shared_ptr const&, DB::MarkRanges const&, std::unordered_map, std::equal_to, std::allocator>> const&, DB::UncompressedCache*, DB::MarkCache*, std::shared_ptr const&, DB::MergeTreeReaderSettings const&, std::map, std::allocator>> const&, std::function const&) const @ 0x000000002c5d1037 12. ./build_docker/./src/Storages/MergeTree/MergeTreeSequentialSource.cpp:169: DB::MergeTreeSequentialSource::MergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, bool, bool, bool) @ 0x000000002c80c330 13. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:35: DB::MergeTreeSequentialSource* std::construct_at[abi:v15007] const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional, bool, bool&, bool&, DB::MergeTreeSequentialSource*>(DB::MergeTreeSequentialSource*, DB::MergeTreeSequentialSourceType&, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr&, std::shared_ptr&, std::vector>&, std::optional&&, bool&&, bool&, bool&) @ 0x000000002c81320f 14. ./contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298: DB::createMergeTreeSequentialSource(DB::MergeTreeSequentialSourceType, DB::MergeTreeData const&, std::shared_ptr const&, std::shared_ptr, std::shared_ptr, std::vector>, std::optional, std::shared_ptr>, bool, bool, bool) @ 0x000000002c810812 15. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:1136: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::createMergedStream() @ 0x000000002c3dd079 16. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x000000002c3d6516 17. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:534: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000002c3e2799 18. ./build_docker/./src/Storages/MergeTree/MergeTask.cpp:0: DB::MergeTask::execute() @ 0x000000002c3ef3fe 19. ./build_docker/./src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:0: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000002cc2f4c0 20. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 21. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 22. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 23. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 24. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 25. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 26. asan_thread_start(void*) @ 0x000000000aa43059 27. ? @ 0x00007fa4c6412ac3 28. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:38.149306 [ 19897 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:38.159734 [ 19897 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:38.561761 [ 19908 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:38.566455 [ 19908 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:39.373892 [ 19898 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:39.375470 [ 19898 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:40.451086 [ 19898 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:40.459962 [ 19898 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:41.354895 [ 19899 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:41.357260 [ 19899 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:41.413973 [ 19958 ] {} DDLWorker(rdb_test_knhvna7w): Error on initialization of rdb_test_knhvna7w: Code: 253. DB::Exception: Replica r1 of shard s1 of replicated database at /test/test_knhvna7w/rdb already exists. Replica host ID: 'fa9ed7cbd16c:9000:e021de81-635c-4962-8732-ca74e4a7f0cb', current host ID: 'fa9ed7cbd16c:9000:901fbf11-3c89-46c1-ba8d-27b10826f797'. (REPLICA_ALREADY_EXISTS), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type, std::type_identity::type, std::type_identity::type>, String&, String&, String&, String&, String&) @ 0x0000000027d8bad5 4. ./build_docker/./src/Databases/DatabaseReplicated.cpp:458: DB::DatabaseReplicated::tryConnectToZooKeeperAndInitDatabase(DB::LoadingStrictnessLevel) @ 0x0000000027d43914 5. ./build_docker/./src/Databases/DatabaseReplicatedWorker.cpp:0: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000027d9c95d 6. ./build_docker/./src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x000000002867281a 7. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000286a04bf 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 9. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 10. asan_thread_start(void*) @ 0x000000000aa43059 11. ? @ 0x00007fa4c6412ac3 12. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:41.854733 [ 19897 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:41.866780 [ 19897 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:42.113987 [ 19894 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:42.116250 [ 19894 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:42.805303 [ 19903 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:42.808165 [ 19903 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:43.117351 [ 19896 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:43.141864 [ 19896 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:44.212678 [ 19907 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:44.215042 [ 19907 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:44.806757 [ 19900 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:44.816222 [ 19900 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:45.331582 [ 19895 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:45.334693 [ 19895 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:46.330331 [ 19906 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} MutatePlainMergeTreeTask: Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK) (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:46.336255 [ 19906 ] {0afe7b0f-62e6-4e2e-9799-1ad478c9be0c::all_1_2_1_4} virtual bool DB::MutatePlainMergeTreeTask::executeStep(): Code: 10. DB::Exception: Not found column _block_offset in block. There are only columns: id, value, event_time, _row_exists. (NOT_FOUND_COLUMN_IN_BLOCK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, String const&, String&&) @ 0x000000000c44a97a 4. ./build_docker/./src/Core/Block.cpp:322: DB::Block::getByName(String const&, bool) const @ 0x00000000271fd6b9 5. ./build_docker/./src/Storages/MergeTree/MergeTreeDataPartWriterWide.cpp:0: DB::MergeTreeDataPartWriterWide::write(DB::Block const&, DB::PODArray, 63ul, 64ul> const*) @ 0x000000002c61a55f 6. ./build_docker/./src/Storages/MergeTree/MergedBlockOutputStream.cpp:368: DB::MergedBlockOutputStream::write(DB::Block const&) @ 0x000000002c8dd257 7. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::mutateOriginalPartAndPrepareProjections() @ 0x000000002c90120e 8. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::PartMergerWriter::execute() @ 0x000000002c946e4a 9. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:1463: DB::MutateAllPartColumnsTask::executeStep() @ 0x000000002c93e3f4 10. ./build_docker/./src/Storages/MergeTree/MutateTask.cpp:0: DB::MutateTask::execute() @ 0x000000002c907bff 11. ./build_docker/./src/Storages/MergeTree/MutatePlainMergeTreeTask.cpp:91: DB::MutatePlainMergeTreeTask::executeStep() @ 0x000000002cc3501a 12. ./build_docker/./src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:298: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000002c427f28 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000002c429675 14. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af7eaf0 15. ./contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x000000001af8a79f 16. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x000000001af79c52 17. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001af86055 18. asan_thread_start(void*) @ 0x000000000aa43059 19. ? @ 0x00007fa4c6412ac3 20. ? @ 0x00007fa4c64a4850 (version 24.8.14.10508.altinitytest (altinity build)) 2025.09.25 21:36:46.532542 [ 19277 ] {} virtual void DB::StorageBuffer::flushAndPrepareForShutdown(): Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=4, jobs=4). (CANNOT_SCHEDULE_TASK), Stack trace (when copying this message, always include the lines below): 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034353454 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb0269 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa8e445 3. ./src/Common/LoggingFormatStringHelpers.h:45: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000001af8b0f7 4. ./contrib/llvm-project/libcxx/include/string:1499: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000001af80b71 5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001af80746 6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x00000000263a174e 7. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000002be5219f 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageBuffer::optimize(std::shared_ptr const&, std::shared_ptr const&, std::shared_ptr const&, bool, bool, std::vector> const&, bool, std::shared_ptr) @ 0x000000002be51b71 9. ./build_docker/./src/Storages/StorageBuffer.cpp:782: DB::StorageBuffer::flushAndPrepareForShutdown() @ 0x000000002be5164e 10. ./contrib/llvm-project/libcxx/include/__tree:189: DB::DatabaseWithOwnTablesBase::shutdown() @ 0x0000000027dbf2b9 11. ./contrib/llvm-project/libcxx/include/__tree:189: DB::DatabaseCatalog::shutdownImpl() @ 0x00000000286aad28 12. ./build_docker/./src/Interpreters/Context.cpp:624: DB::ContextSharedPart::shutdown() @ 0x0000000028589078 13. ./build_docker/./programs/server/Server.cpp:0: BasicScopeGuard> const&)::$_2>::~BasicScopeGuard() @ 0x000000001b188e32 14. ./build_docker/./programs/server/Server.cpp:2323: DB::Server::main(std::vector> const&) @ 0x000000001b16ecf4 15. ./build_docker/./base/poco/Util/src/Application.cpp:0: Poco::Util::Application::run() @ 0x0000000034593cf7 16. ./build_docker/./programs/server/Server.cpp:396: DB::Server::run() @ 0x000000001b14b97a 17. ./build_docker/./programs/server/Server.cpp:0: mainEntryClickHouseServer(int, char**) @ 0x000000001b146de2 18. ./build_docker/./programs/main.cpp:0: main @ 0x000000000aa7c909 19. ? @ 0x00007fa4c63a7d90 20. ? @ 0x00007fa4c63a7e40 21. _start @ 0x000000000a9ad02e (version 24.8.14.10508.altinitytest (altinity build))